[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 991 - Still Failing

[Apache Jenkins Server <je...@builds.apache.org>]

Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/991/

1 tests failed.
FAILED:  org.apache.lucene.index.TestDuelingCodecsAtNight.testBigEquals

Error Message:
Java heap space

Stack Trace:
java.lang.OutOfMemoryError: Java heap space
	at __randomizedtesting.SeedInfo.seed([D2641310986A09D9:316048953371CD9E]:0)
	at org.apache.lucene.util.fst.BytesStore.skipBytes(BytesStore.java:309)
	at org.apache.lucene.util.fst.FST.addNode(FST.java:792)
	at org.apache.lucene.util.fst.NodeHash.add(NodeHash.java:126)
	at org.apache.lucene.util.fst.Builder.compileNode(Builder.java:215)
	at org.apache.lucene.util.fst.Builder.freezeTail(Builder.java:310)
	at org.apache.lucene.util.fst.Builder.add(Builder.java:417)
	at org.apache.lucene.codecs.memory.MemoryPostingsFormat$TermsWriter.finishTerm(MemoryPostingsFormat.java:258)
	at org.apache.lucene.codecs.memory.MemoryPostingsFormat$TermsWriter.access$500(MemoryPostingsFormat.java:113)
	at org.apache.lucene.codecs.memory.MemoryPostingsFormat$MemoryFieldsConsumer.write(MemoryPostingsFormat.java:400)
	at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsWriter.write(PerFieldPostingsFormat.java:198)
	at org.apache.lucene.codecs.FieldsConsumer.merge(FieldsConsumer.java:105)
	at org.apache.lucene.index.SegmentMerger.mergeTerms(SegmentMerger.java:193)
	at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:95)
	at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4068)
	at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3648)
	at org.apache.lucene.index.SerialMergeScheduler.merge(SerialMergeScheduler.java:40)
	at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:1929)
	at org.apache.lucene.index.IndexWriter.doAfterSegmentFlushed(IndexWriter.java:4712)
	at org.apache.lucene.index.DocumentsWriter$MergePendingEvent.process(DocumentsWriter.java:695)
	at org.apache.lucene.index.IndexWriter.processEvents(IndexWriter.java:4738)
	at org.apache.lucene.index.IndexWriter.processEvents(IndexWriter.java:4729)
	at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1476)
	at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1254)
	at org.apache.lucene.index.RandomIndexWriter.addDocument(RandomIndexWriter.java:173)
	at org.apache.lucene.index.TestDuelingCodecs.createRandomIndex(TestDuelingCodecs.java:139)
	at org.apache.lucene.index.TestDuelingCodecsAtNight.testBigEquals(TestDuelingCodecsAtNight.java:34)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1665)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:864)




Build Log:
[...truncated 1524 lines...]
   [junit4] Suite: org.apache.lucene.index.TestDuelingCodecsAtNight
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestDuelingCodecsAtNight -Dtests.method=testBigEquals -Dtests.seed=D2641310986A09D9 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=pt -Dtests.timezone=America/Santarem -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR    466s J1 | TestDuelingCodecsAtNight.testBigEquals <<<
   [junit4]    > Throwable #1: java.lang.OutOfMemoryError: Java heap space
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([D2641310986A09D9:316048953371CD9E]:0)
   [junit4]    > 	at org.apache.lucene.util.fst.BytesStore.skipBytes(BytesStore.java:309)
   [junit4]    > 	at org.apache.lucene.util.fst.FST.addNode(FST.java:792)
   [junit4]    > 	at org.apache.lucene.util.fst.NodeHash.add(NodeHash.java:126)
   [junit4]    > 	at org.apache.lucene.util.fst.Builder.compileNode(Builder.java:215)
   [junit4]    > 	at org.apache.lucene.util.fst.Builder.freezeTail(Builder.java:310)
   [junit4]    > 	at org.apache.lucene.util.fst.Builder.add(Builder.java:417)
   [junit4]    > 	at org.apache.lucene.codecs.memory.MemoryPostingsFormat$TermsWriter.finishTerm(MemoryPostingsFormat.java:258)
   [junit4]    > 	at org.apache.lucene.codecs.memory.MemoryPostingsFormat$TermsWriter.access$500(MemoryPostingsFormat.java:113)
   [junit4]    > 	at org.apache.lucene.codecs.memory.MemoryPostingsFormat$MemoryFieldsConsumer.write(MemoryPostingsFormat.java:400)
   [junit4]    > 	at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsWriter.write(PerFieldPostingsFormat.java:198)
   [junit4]    > 	at org.apache.lucene.codecs.FieldsConsumer.merge(FieldsConsumer.java:105)
   [junit4]    > 	at org.apache.lucene.index.SegmentMerger.mergeTerms(SegmentMerger.java:193)
   [junit4]    > 	at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:95)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4068)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3648)
   [junit4]    > 	at org.apache.lucene.index.SerialMergeScheduler.merge(SerialMergeScheduler.java:40)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:1929)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.doAfterSegmentFlushed(IndexWriter.java:4712)
   [junit4]    > 	at org.apache.lucene.index.DocumentsWriter$MergePendingEvent.process(DocumentsWriter.java:695)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.processEvents(IndexWriter.java:4738)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.processEvents(IndexWriter.java:4729)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1476)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1254)
   [junit4]    > 	at org.apache.lucene.index.RandomIndexWriter.addDocument(RandomIndexWriter.java:173)
   [junit4]    > 	at org.apache.lucene.index.TestDuelingCodecs.createRandomIndex(TestDuelingCodecs.java:139)
   [junit4]    > 	at org.apache.lucene.index.TestDuelingCodecsAtNight.testBigEquals(TestDuelingCodecsAtNight.java:34)
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/test/J1/temp/lucene.index.TestDuelingCodecsAtNight_D2641310986A09D9-001
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene53): {}, docValues:{}, sim=DefaultSimilarity, locale=pt, timezone=America/Santarem
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_72 (64-bit)/cpus=4,threads=1,free=111720776,total=435159040
   [junit4]   2> NOTE: All tests run in this JVM: [TestFieldMaskingSpanQuery, TestPerFieldDocValuesFormat, TestIndexInput, TestMultiTermQueryRewrites, TestCachingTokenFilter, TestFSTs, TestDocument, TestTransactionRollback, TestIndexReaderClose, TestSortRescorer, TestMultiCollector, TestIntBlockPool, TestCrash, TestLockFactory, TestLucene50SegmentInfoFormat, TestUsageTrackingFilterCachingPolicy, TestDocumentsWriterStallControl, TestDocCount, TestIndexWriterOnDiskFull, TestUnicodeUtil, TestLazyProxSkipping, TestSegmentMerger, TestFixedBitSet, TestDateSort, TestCloseableThreadLocal, TestSpanOrQuery, TestMultiTermsEnum, TestWeakIdentityMap, TestBoostQuery, TestLucene50FieldInfoFormat, TestMergeSchedulerExternal, TestPackedInts, TestSmallFloat, TestDemoParallelLeafReader, TestTermRangeFilter, TestFilteredSearch, TestBooleanQueryVisitSubscorers, TestFieldValueQuery, TestFlushByRamOrCountsPolicy, TestFieldCacheRewriteMethod, MultiCollectorTest, TestFastDecompressionMode, TestCheckIndex, TestSpansAdvanced2, TestNewestSegment, TestScoreCachingWrappingScorer, TestMathUtil, TestTermRangeQuery, TestIndexWriterExceptions, TestSortedNumericSortField, TestIndexSearcher, TestFieldValueFilter, TestRecyclingByteBlockAllocator, TestFieldReuse, TestCodecs, TestExitableDirectoryReader, TestOmitPositions, TestBlockPostingsFormat3, TestExceedMaxTermLength, TestSearchAfter, TestSameScoresWithThreads, TestIntArrayDocIdSet, TestBoolean2, TestLogMergePolicy, TestStringHelper, TestAllFilesHaveCodecHeader, TestTermScorer, TestFilterDirectoryReader, TestFlex, TestLucene50StoredFieldsFormat, TestLucene50StoredFieldsFormatHighCompression, TestFieldType, Test2BSortedDocValuesFixedSorted, Test2BSortedDocValuesOrds, TestAllFilesHaveChecksumFooter, TestCodecUtil, TestDocInverterPerFieldErrorInfo, TestDocValues, TestDuelingCodecsAtNight]
   [junit4] Completed [359/411] on J1 in 487.97s, 3 tests, 1 error <<< FAILURES!

[...truncated 185 lines...]
   [junit4] JVM J1: stdout was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/test/temp/junit4-J1-20151019_160522_549.sysout
   [junit4] >>> JVM J1: stdout (verbatim) ----
   [junit4] java.lang.OutOfMemoryError: Java heap space
   [junit4] Dumping heap to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/heapdumps/java_pid6468.hprof ...
   [junit4] Heap dump file created [422116720 bytes in 0.619 secs]
   [junit4] <<< JVM J1: EOF ----

[...truncated 10 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build.xml:50: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 411 suites (1 ignored), 3311 tests, 1 error, 30 ignored (26 assumptions) [seed: D2641310986A09D9]

Total time: 50 minutes 40 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1045 - Still Failing

[Apache Jenkins Server <je...@builds.apache.org>]

Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1045/

10 tests failed.
FAILED:  org.apache.lucene.index.TestIndexWriterThreadsToSegments.testSegmentCountOnFlushRandom

Error Message:
Test abandoned because suite timeout was reached.

Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
	at __randomizedtesting.SeedInfo.seed([F72A04E52535E596]:0)


FAILED:  junit.framework.TestSuite.org.apache.lucene.index.TestIndexWriterThreadsToSegments

Error Message:
Suite timeout exceeded (>= 7200000 msec).

Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
	at __randomizedtesting.SeedInfo.seed([F72A04E52535E596]:0)


FAILED:  junit.framework.TestSuite.org.apache.lucene.index.TestIndexWriterThreadsToSegments

Error Message:
Captured an uncaught exception in thread: Thread[id=1721, name=Thread-1644, state=RUNNABLE, group=TGRP-TestIndexWriterThreadsToSegments]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=1721, name=Thread-1644, state=RUNNABLE, group=TGRP-TestIndexWriterThreadsToSegments]
Caused by: java.lang.RuntimeException: java.nio.file.FileSystemException: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/test/J1/temp/lucene.index.TestIndexWriterThreadsToSegments_F72A04E52535E596-001/tempDir-001/_1ks.fdx: Too many open files
	at __randomizedtesting.SeedInfo.seed([F72A04E52535E596]:0)
	at org.apache.lucene.index.TestIndexWriterThreadsToSegments$2.run(TestIndexWriterThreadsToSegments.java:218)
Caused by: java.nio.file.FileSystemException: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/test/J1/temp/lucene.index.TestIndexWriterThreadsToSegments_F72A04E52535E596-001/tempDir-001/_1ks.fdx: Too many open files
	at org.apache.lucene.mockfile.HandleLimitFS.onOpen(HandleLimitFS.java:49)
	at org.apache.lucene.mockfile.HandleTrackingFS.callOpenHook(HandleTrackingFS.java:82)
	at org.apache.lucene.mockfile.HandleTrackingFS.newOutputStream(HandleTrackingFS.java:161)
	at org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:198)
	at java.nio.file.Files.newOutputStream(Files.java:172)
	at org.apache.lucene.store.FSDirectory$FSIndexOutput.<init>(FSDirectory.java:276)
	at org.apache.lucene.store.FSDirectory.createOutput(FSDirectory.java:225)
	at org.apache.lucene.store.MockDirectoryWrapper.createOutput(MockDirectoryWrapper.java:593)
	at org.apache.lucene.store.LockValidatingDirectoryWrapper.createOutput(LockValidatingDirectoryWrapper.java:44)
	at org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:43)
	at org.apache.lucene.codecs.compressing.CompressingStoredFieldsWriter.<init>(CompressingStoredFieldsWriter.java:108)
	at org.apache.lucene.codecs.compressing.CompressingStoredFieldsFormat.fieldsWriter(CompressingStoredFieldsFormat.java:128)
	at org.apache.lucene.codecs.lucene50.Lucene50StoredFieldsFormat.fieldsWriter(Lucene50StoredFieldsFormat.java:183)
	at org.apache.lucene.codecs.asserting.AssertingStoredFieldsFormat.fieldsWriter(AssertingStoredFieldsFormat.java:49)
	at org.apache.lucene.index.DefaultIndexingChain.initStoredFieldsWriter(DefaultIndexingChain.java:81)
	at org.apache.lucene.index.DefaultIndexingChain.startStoredFields(DefaultIndexingChain.java:258)
	at org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:295)
	at org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:234)
	at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:450)
	at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1477)
	at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1256)
	at org.apache.lucene.index.TestIndexWriterThreadsToSegments$2.run(TestIndexWriterThreadsToSegments.java:207)


FAILED:  junit.framework.TestSuite.org.apache.lucene.index.TestIndexWriterThreadsToSegments

Error Message:
Captured an uncaught exception in thread: Thread[id=1719, name=Thread-1642, state=RUNNABLE, group=TGRP-TestIndexWriterThreadsToSegments]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=1719, name=Thread-1642, state=RUNNABLE, group=TGRP-TestIndexWriterThreadsToSegments]
Caused by: java.lang.RuntimeException: org.apache.lucene.store.AlreadyClosedException: this IndexWriter is closed
	at __randomizedtesting.SeedInfo.seed([F72A04E52535E596]:0)
	at org.apache.lucene.index.TestIndexWriterThreadsToSegments$2.run(TestIndexWriterThreadsToSegments.java:218)
Caused by: org.apache.lucene.store.AlreadyClosedException: this IndexWriter is closed
	at org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:720)
	at org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:734)
	at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1473)
	at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1256)
	at org.apache.lucene.index.TestIndexWriterThreadsToSegments$2.run(TestIndexWriterThreadsToSegments.java:207)
Caused by: java.nio.file.FileSystemException: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/test/J1/temp/lucene.index.TestIndexWriterThreadsToSegments_F72A04E52535E596-001/tempDir-001/_1ku.fdx: Too many open files
	at org.apache.lucene.mockfile.HandleLimitFS.onOpen(HandleLimitFS.java:49)
	at org.apache.lucene.mockfile.HandleTrackingFS.callOpenHook(HandleTrackingFS.java:82)
	at org.apache.lucene.mockfile.HandleTrackingFS.newOutputStream(HandleTrackingFS.java:161)
	at org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:198)
	at java.nio.file.Files.newOutputStream(Files.java:172)
	at org.apache.lucene.store.FSDirectory$FSIndexOutput.<init>(FSDirectory.java:276)
	at org.apache.lucene.store.FSDirectory.createOutput(FSDirectory.java:225)
	at org.apache.lucene.store.MockDirectoryWrapper.createOutput(MockDirectoryWrapper.java:593)
	at org.apache.lucene.store.LockValidatingDirectoryWrapper.createOutput(LockValidatingDirectoryWrapper.java:44)
	at org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:43)
	at org.apache.lucene.codecs.compressing.CompressingStoredFieldsWriter.<init>(CompressingStoredFieldsWriter.java:108)
	at org.apache.lucene.codecs.compressing.CompressingStoredFieldsFormat.fieldsWriter(CompressingStoredFieldsFormat.java:128)
	at org.apache.lucene.codecs.lucene50.Lucene50StoredFieldsFormat.fieldsWriter(Lucene50StoredFieldsFormat.java:183)
	at org.apache.lucene.codecs.asserting.AssertingStoredFieldsFormat.fieldsWriter(AssertingStoredFieldsFormat.java:49)
	at org.apache.lucene.index.DefaultIndexingChain.initStoredFieldsWriter(DefaultIndexingChain.java:81)
	at org.apache.lucene.index.DefaultIndexingChain.startStoredFields(DefaultIndexingChain.java:258)
	at org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:295)
	at org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:234)
	at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:450)
	at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1477)
	... 2 more


FAILED:  junit.framework.TestSuite.org.apache.lucene.index.TestIndexWriterThreadsToSegments

Error Message:
Captured an uncaught exception in thread: Thread[id=1722, name=Thread-1645, state=RUNNABLE, group=TGRP-TestIndexWriterThreadsToSegments]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=1722, name=Thread-1645, state=RUNNABLE, group=TGRP-TestIndexWriterThreadsToSegments]
Caused by: java.lang.RuntimeException: java.nio.file.FileSystemException: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/test/J1/temp/lucene.index.TestIndexWriterThreadsToSegments_F72A04E52535E596-001/tempDir-001/_1kp.fdx: Too many open files
	at __randomizedtesting.SeedInfo.seed([F72A04E52535E596]:0)
	at org.apache.lucene.index.TestIndexWriterThreadsToSegments$2.run(TestIndexWriterThreadsToSegments.java:218)
Caused by: java.nio.file.FileSystemException: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/test/J1/temp/lucene.index.TestIndexWriterThreadsToSegments_F72A04E52535E596-001/tempDir-001/_1kp.fdx: Too many open files
	at org.apache.lucene.mockfile.HandleLimitFS.onOpen(HandleLimitFS.java:49)
	at org.apache.lucene.mockfile.HandleTrackingFS.callOpenHook(HandleTrackingFS.java:82)
	at org.apache.lucene.mockfile.HandleTrackingFS.newOutputStream(HandleTrackingFS.java:161)
	at org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:198)
	at java.nio.file.Files.newOutputStream(Files.java:172)
	at org.apache.lucene.store.FSDirectory$FSIndexOutput.<init>(FSDirectory.java:276)
	at org.apache.lucene.store.FSDirectory.createOutput(FSDirectory.java:225)
	at org.apache.lucene.store.MockDirectoryWrapper.createOutput(MockDirectoryWrapper.java:593)
	at org.apache.lucene.store.LockValidatingDirectoryWrapper.createOutput(LockValidatingDirectoryWrapper.java:44)
	at org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:43)
	at org.apache.lucene.codecs.compressing.CompressingStoredFieldsWriter.<init>(CompressingStoredFieldsWriter.java:108)
	at org.apache.lucene.codecs.compressing.CompressingStoredFieldsFormat.fieldsWriter(CompressingStoredFieldsFormat.java:128)
	at org.apache.lucene.codecs.lucene50.Lucene50StoredFieldsFormat.fieldsWriter(Lucene50StoredFieldsFormat.java:183)
	at org.apache.lucene.codecs.asserting.AssertingStoredFieldsFormat.fieldsWriter(AssertingStoredFieldsFormat.java:49)
	at org.apache.lucene.index.DefaultIndexingChain.initStoredFieldsWriter(DefaultIndexingChain.java:81)
	at org.apache.lucene.index.DefaultIndexingChain.startStoredFields(DefaultIndexingChain.java:258)
	at org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:295)
	at org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:234)
	at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:450)
	at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1477)
	at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1256)
	at org.apache.lucene.index.TestIndexWriterThreadsToSegments$2.run(TestIndexWriterThreadsToSegments.java:207)


FAILED:  junit.framework.TestSuite.org.apache.lucene.index.TestIndexWriterThreadsToSegments

Error Message:
Captured an uncaught exception in thread: Thread[id=1723, name=Thread-1646, state=RUNNABLE, group=TGRP-TestIndexWriterThreadsToSegments]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=1723, name=Thread-1646, state=RUNNABLE, group=TGRP-TestIndexWriterThreadsToSegments]
Caused by: java.lang.RuntimeException: java.nio.file.FileSystemException: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/test/J1/temp/lucene.index.TestIndexWriterThreadsToSegments_F72A04E52535E596-001/tempDir-001/_1kt.fdt: Too many open files
	at __randomizedtesting.SeedInfo.seed([F72A04E52535E596]:0)
	at org.apache.lucene.index.TestIndexWriterThreadsToSegments$2.run(TestIndexWriterThreadsToSegments.java:218)
Caused by: java.nio.file.FileSystemException: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/test/J1/temp/lucene.index.TestIndexWriterThreadsToSegments_F72A04E52535E596-001/tempDir-001/_1kt.fdt: Too many open files
	at org.apache.lucene.mockfile.HandleLimitFS.onOpen(HandleLimitFS.java:49)
	at org.apache.lucene.mockfile.HandleTrackingFS.callOpenHook(HandleTrackingFS.java:82)
	at org.apache.lucene.mockfile.HandleTrackingFS.newOutputStream(HandleTrackingFS.java:161)
	at org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:198)
	at java.nio.file.Files.newOutputStream(Files.java:172)
	at org.apache.lucene.store.FSDirectory$FSIndexOutput.<init>(FSDirectory.java:276)
	at org.apache.lucene.store.FSDirectory.createOutput(FSDirectory.java:225)
	at org.apache.lucene.store.MockDirectoryWrapper.createOutput(MockDirectoryWrapper.java:593)
	at org.apache.lucene.store.LockValidatingDirectoryWrapper.createOutput(LockValidatingDirectoryWrapper.java:44)
	at org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:43)
	at org.apache.lucene.codecs.compressing.CompressingStoredFieldsWriter.<init>(CompressingStoredFieldsWriter.java:111)
	at org.apache.lucene.codecs.compressing.CompressingStoredFieldsFormat.fieldsWriter(CompressingStoredFieldsFormat.java:128)
	at org.apache.lucene.codecs.lucene50.Lucene50StoredFieldsFormat.fieldsWriter(Lucene50StoredFieldsFormat.java:183)
	at org.apache.lucene.codecs.asserting.AssertingStoredFieldsFormat.fieldsWriter(AssertingStoredFieldsFormat.java:49)
	at org.apache.lucene.index.DefaultIndexingChain.initStoredFieldsWriter(DefaultIndexingChain.java:81)
	at org.apache.lucene.index.DefaultIndexingChain.startStoredFields(DefaultIndexingChain.java:258)
	at org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:295)
	at org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:234)
	at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:450)
	at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1477)
	at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1256)
	at org.apache.lucene.index.TestIndexWriterThreadsToSegments$2.run(TestIndexWriterThreadsToSegments.java:207)


FAILED:  junit.framework.TestSuite.org.apache.lucene.index.TestIndexWriterThreadsToSegments

Error Message:
Captured an uncaught exception in thread: Thread[id=1717, name=Thread-1640, state=RUNNABLE, group=TGRP-TestIndexWriterThreadsToSegments]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=1717, name=Thread-1640, state=RUNNABLE, group=TGRP-TestIndexWriterThreadsToSegments]
Caused by: java.lang.RuntimeException: org.apache.lucene.store.AlreadyClosedException: this IndexWriter is closed
	at __randomizedtesting.SeedInfo.seed([F72A04E52535E596]:0)
	at org.apache.lucene.index.TestIndexWriterThreadsToSegments$2.run(TestIndexWriterThreadsToSegments.java:218)
Caused by: org.apache.lucene.store.AlreadyClosedException: this IndexWriter is closed
	at org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:720)
	at org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:734)
	at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1473)
	at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1256)
	at org.apache.lucene.index.TestIndexWriterThreadsToSegments$2.run(TestIndexWriterThreadsToSegments.java:207)
Caused by: java.nio.file.FileSystemException: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/test/J1/temp/lucene.index.TestIndexWriterThreadsToSegments_F72A04E52535E596-001/tempDir-001/_1ku.fdx: Too many open files
	at org.apache.lucene.mockfile.HandleLimitFS.onOpen(HandleLimitFS.java:49)
	at org.apache.lucene.mockfile.HandleTrackingFS.callOpenHook(HandleTrackingFS.java:82)
	at org.apache.lucene.mockfile.HandleTrackingFS.newOutputStream(HandleTrackingFS.java:161)
	at org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:198)
	at java.nio.file.Files.newOutputStream(Files.java:172)
	at org.apache.lucene.store.FSDirectory$FSIndexOutput.<init>(FSDirectory.java:276)
	at org.apache.lucene.store.FSDirectory.createOutput(FSDirectory.java:225)
	at org.apache.lucene.store.MockDirectoryWrapper.createOutput(MockDirectoryWrapper.java:593)
	at org.apache.lucene.store.LockValidatingDirectoryWrapper.createOutput(LockValidatingDirectoryWrapper.java:44)
	at org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:43)
	at org.apache.lucene.codecs.compressing.CompressingStoredFieldsWriter.<init>(CompressingStoredFieldsWriter.java:108)
	at org.apache.lucene.codecs.compressing.CompressingStoredFieldsFormat.fieldsWriter(CompressingStoredFieldsFormat.java:128)
	at org.apache.lucene.codecs.lucene50.Lucene50StoredFieldsFormat.fieldsWriter(Lucene50StoredFieldsFormat.java:183)
	at org.apache.lucene.codecs.asserting.AssertingStoredFieldsFormat.fieldsWriter(AssertingStoredFieldsFormat.java:49)
	at org.apache.lucene.index.DefaultIndexingChain.initStoredFieldsWriter(DefaultIndexingChain.java:81)
	at org.apache.lucene.index.DefaultIndexingChain.startStoredFields(DefaultIndexingChain.java:258)
	at org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:295)
	at org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:234)
	at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:450)
	at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1477)
	... 2 more


FAILED:  junit.framework.TestSuite.org.apache.lucene.index.TestIndexWriterThreadsToSegments

Error Message:
Captured an uncaught exception in thread: Thread[id=1720, name=Thread-1643, state=RUNNABLE, group=TGRP-TestIndexWriterThreadsToSegments]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=1720, name=Thread-1643, state=RUNNABLE, group=TGRP-TestIndexWriterThreadsToSegments]
Caused by: java.lang.RuntimeException: org.apache.lucene.store.AlreadyClosedException: this IndexWriter is closed
	at __randomizedtesting.SeedInfo.seed([F72A04E52535E596]:0)
	at org.apache.lucene.index.TestIndexWriterThreadsToSegments$2.run(TestIndexWriterThreadsToSegments.java:218)
Caused by: org.apache.lucene.store.AlreadyClosedException: this IndexWriter is closed
	at org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:720)
	at org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:734)
	at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1473)
	at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1256)
	at org.apache.lucene.index.TestIndexWriterThreadsToSegments$2.run(TestIndexWriterThreadsToSegments.java:207)
Caused by: java.nio.file.FileSystemException: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/test/J1/temp/lucene.index.TestIndexWriterThreadsToSegments_F72A04E52535E596-001/tempDir-001/_1ku.fdx: Too many open files
	at org.apache.lucene.mockfile.HandleLimitFS.onOpen(HandleLimitFS.java:49)
	at org.apache.lucene.mockfile.HandleTrackingFS.callOpenHook(HandleTrackingFS.java:82)
	at org.apache.lucene.mockfile.HandleTrackingFS.newOutputStream(HandleTrackingFS.java:161)
	at org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:198)
	at java.nio.file.Files.newOutputStream(Files.java:172)
	at org.apache.lucene.store.FSDirectory$FSIndexOutput.<init>(FSDirectory.java:276)
	at org.apache.lucene.store.FSDirectory.createOutput(FSDirectory.java:225)
	at org.apache.lucene.store.MockDirectoryWrapper.createOutput(MockDirectoryWrapper.java:593)
	at org.apache.lucene.store.LockValidatingDirectoryWrapper.createOutput(LockValidatingDirectoryWrapper.java:44)
	at org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:43)
	at org.apache.lucene.codecs.compressing.CompressingStoredFieldsWriter.<init>(CompressingStoredFieldsWriter.java:108)
	at org.apache.lucene.codecs.compressing.CompressingStoredFieldsFormat.fieldsWriter(CompressingStoredFieldsFormat.java:128)
	at org.apache.lucene.codecs.lucene50.Lucene50StoredFieldsFormat.fieldsWriter(Lucene50StoredFieldsFormat.java:183)
	at org.apache.lucene.codecs.asserting.AssertingStoredFieldsFormat.fieldsWriter(AssertingStoredFieldsFormat.java:49)
	at org.apache.lucene.index.DefaultIndexingChain.initStoredFieldsWriter(DefaultIndexingChain.java:81)
	at org.apache.lucene.index.DefaultIndexingChain.startStoredFields(DefaultIndexingChain.java:258)
	at org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:295)
	at org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:234)
	at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:450)
	at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1477)
	... 2 more


FAILED:  junit.framework.TestSuite.org.apache.lucene.index.TestIndexWriterThreadsToSegments

Error Message:
Captured an uncaught exception in thread: Thread[id=1718, name=Thread-1641, state=RUNNABLE, group=TGRP-TestIndexWriterThreadsToSegments]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=1718, name=Thread-1641, state=RUNNABLE, group=TGRP-TestIndexWriterThreadsToSegments]
Caused by: java.lang.RuntimeException: org.apache.lucene.store.AlreadyClosedException: this IndexWriter is closed
	at __randomizedtesting.SeedInfo.seed([F72A04E52535E596]:0)
	at org.apache.lucene.index.TestIndexWriterThreadsToSegments$2.run(TestIndexWriterThreadsToSegments.java:218)
Caused by: org.apache.lucene.store.AlreadyClosedException: this IndexWriter is closed
	at org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:720)
	at org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:734)
	at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1473)
	at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1256)
	at org.apache.lucene.index.TestIndexWriterThreadsToSegments$2.run(TestIndexWriterThreadsToSegments.java:207)
Caused by: java.nio.file.FileSystemException: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/test/J1/temp/lucene.index.TestIndexWriterThreadsToSegments_F72A04E52535E596-001/tempDir-001/_1ku.fdx: Too many open files
	at org.apache.lucene.mockfile.HandleLimitFS.onOpen(HandleLimitFS.java:49)
	at org.apache.lucene.mockfile.HandleTrackingFS.callOpenHook(HandleTrackingFS.java:82)
	at org.apache.lucene.mockfile.HandleTrackingFS.newOutputStream(HandleTrackingFS.java:161)
	at org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:198)
	at java.nio.file.Files.newOutputStream(Files.java:172)
	at org.apache.lucene.store.FSDirectory$FSIndexOutput.<init>(FSDirectory.java:276)
	at org.apache.lucene.store.FSDirectory.createOutput(FSDirectory.java:225)
	at org.apache.lucene.store.MockDirectoryWrapper.createOutput(MockDirectoryWrapper.java:593)
	at org.apache.lucene.store.LockValidatingDirectoryWrapper.createOutput(LockValidatingDirectoryWrapper.java:44)
	at org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:43)
	at org.apache.lucene.codecs.compressing.CompressingStoredFieldsWriter.<init>(CompressingStoredFieldsWriter.java:108)
	at org.apache.lucene.codecs.compressing.CompressingStoredFieldsFormat.fieldsWriter(CompressingStoredFieldsFormat.java:128)
	at org.apache.lucene.codecs.lucene50.Lucene50StoredFieldsFormat.fieldsWriter(Lucene50StoredFieldsFormat.java:183)
	at org.apache.lucene.codecs.asserting.AssertingStoredFieldsFormat.fieldsWriter(AssertingStoredFieldsFormat.java:49)
	at org.apache.lucene.index.DefaultIndexingChain.initStoredFieldsWriter(DefaultIndexingChain.java:81)
	at org.apache.lucene.index.DefaultIndexingChain.startStoredFields(DefaultIndexingChain.java:258)
	at org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:295)
	at org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:234)
	at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:450)
	at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1477)
	... 2 more


FAILED:  junit.framework.TestSuite.org.apache.lucene.index.TestIndexWriterThreadsToSegments

Error Message:
Captured an uncaught exception in thread: Thread[id=1714, name=Thread-1637, state=RUNNABLE, group=TGRP-TestIndexWriterThreadsToSegments]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=1714, name=Thread-1637, state=RUNNABLE, group=TGRP-TestIndexWriterThreadsToSegments]
Caused by: java.lang.RuntimeException: java.nio.file.FileSystemException: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/test/J1/temp/lucene.index.TestIndexWriterThreadsToSegments_F72A04E52535E596-001/tempDir-001/_1ku.fdx: Too many open files
	at __randomizedtesting.SeedInfo.seed([F72A04E52535E596]:0)
	at org.apache.lucene.index.TestIndexWriterThreadsToSegments$2.run(TestIndexWriterThreadsToSegments.java:218)
Caused by: java.nio.file.FileSystemException: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/test/J1/temp/lucene.index.TestIndexWriterThreadsToSegments_F72A04E52535E596-001/tempDir-001/_1ku.fdx: Too many open files
	at org.apache.lucene.mockfile.HandleLimitFS.onOpen(HandleLimitFS.java:49)
	at org.apache.lucene.mockfile.HandleTrackingFS.callOpenHook(HandleTrackingFS.java:82)
	at org.apache.lucene.mockfile.HandleTrackingFS.newOutputStream(HandleTrackingFS.java:161)
	at org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:198)
	at java.nio.file.Files.newOutputStream(Files.java:172)
	at org.apache.lucene.store.FSDirectory$FSIndexOutput.<init>(FSDirectory.java:276)
	at org.apache.lucene.store.FSDirectory.createOutput(FSDirectory.java:225)
	at org.apache.lucene.store.MockDirectoryWrapper.createOutput(MockDirectoryWrapper.java:593)
	at org.apache.lucene.store.LockValidatingDirectoryWrapper.createOutput(LockValidatingDirectoryWrapper.java:44)
	at org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:43)
	at org.apache.lucene.codecs.compressing.CompressingStoredFieldsWriter.<init>(CompressingStoredFieldsWriter.java:108)
	at org.apache.lucene.codecs.compressing.CompressingStoredFieldsFormat.fieldsWriter(CompressingStoredFieldsFormat.java:128)
	at org.apache.lucene.codecs.lucene50.Lucene50StoredFieldsFormat.fieldsWriter(Lucene50StoredFieldsFormat.java:183)
	at org.apache.lucene.codecs.asserting.AssertingStoredFieldsFormat.fieldsWriter(AssertingStoredFieldsFormat.java:49)
	at org.apache.lucene.index.DefaultIndexingChain.initStoredFieldsWriter(DefaultIndexingChain.java:81)
	at org.apache.lucene.index.DefaultIndexingChain.startStoredFields(DefaultIndexingChain.java:258)
	at org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:295)
	at org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:234)
	at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:450)
	at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1477)
	at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1256)
	at org.apache.lucene.index.TestIndexWriterThreadsToSegments$2.run(TestIndexWriterThreadsToSegments.java:207)




Build Log:
[...truncated 1678 lines...]
   [junit4] Suite: org.apache.lucene.index.TestIndexWriterThreadsToSegments
   [junit4]   2> dec 12, 2015 2:10:10 PM com.carrotsearch.randomizedtesting.RandomizedRunner$QueueUncaughtExceptionsHandler uncaughtException
   [junit4]   2> WARNING: Uncaught exception in thread: Thread[Thread-1642,5,TGRP-TestIndexWriterThreadsToSegments]
   [junit4]   2> java.lang.RuntimeException: org.apache.lucene.store.AlreadyClosedException: this IndexWriter is closed
   [junit4]   2> 	at __randomizedtesting.SeedInfo.seed([F72A04E52535E596]:0)
   [junit4]   2> 	at org.apache.lucene.index.TestIndexWriterThreadsToSegments$2.run(TestIndexWriterThreadsToSegments.java:218)
   [junit4]   2> Caused by: org.apache.lucene.store.AlreadyClosedException: this IndexWriter is closed
   [junit4]   2> 	at org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:720)
   [junit4]   2> 	at org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:734)
   [junit4]   2> 	at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1473)
   [junit4]   2> 	at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1256)
   [junit4]   2> 	at org.apache.lucene.index.TestIndexWriterThreadsToSegments$2.run(TestIndexWriterThreadsToSegments.java:207)
   [junit4]   2> Caused by: java.nio.file.FileSystemException: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/test/J1/temp/lucene.index.TestIndexWriterThreadsToSegments_F72A04E52535E596-001/tempDir-001/_1ku.fdx: Too many open files
   [junit4]   2> 	at org.apache.lucene.mockfile.HandleLimitFS.onOpen(HandleLimitFS.java:49)
   [junit4]   2> 	at org.apache.lucene.mockfile.HandleTrackingFS.callOpenHook(HandleTrackingFS.java:82)
   [junit4]   2> 	at org.apache.lucene.mockfile.HandleTrackingFS.newOutputStream(HandleTrackingFS.java:161)
   [junit4]   2> 	at org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:198)
   [junit4]   2> 	at java.nio.file.Files.newOutputStream(Files.java:172)
   [junit4]   2> 	at org.apache.lucene.store.FSDirectory$FSIndexOutput.<init>(FSDirectory.java:276)
   [junit4]   2> 	at org.apache.lucene.store.FSDirectory.createOutput(FSDirectory.java:225)
   [junit4]   2> 	at org.apache.lucene.store.MockDirectoryWrapper.createOutput(MockDirectoryWrapper.java:593)
   [junit4]   2> 	at org.apache.lucene.store.LockValidatingDirectoryWrapper.createOutput(LockValidatingDirectoryWrapper.java:44)
   [junit4]   2> 	at org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:43)
   [junit4]   2> 	at org.apache.lucene.codecs.compressing.CompressingStoredFieldsWriter.<init>(CompressingStoredFieldsWriter.java:108)
   [junit4]   2> 	at org.apache.lucene.codecs.compressing.CompressingStoredFieldsFormat.fieldsWriter(CompressingStoredFieldsFormat.java:128)
   [junit4]   2> 	at org.apache.lucene.codecs.lucene50.Lucene50StoredFieldsFormat.fieldsWriter(Lucene50StoredFieldsFormat.java:183)
   [junit4]   2> 	at org.apache.lucene.codecs.asserting.AssertingStoredFieldsFormat.fieldsWriter(AssertingStoredFieldsFormat.java:49)
   [junit4]   2> 	at org.apache.lucene.index.DefaultIndexingChain.initStoredFieldsWriter(DefaultIndexingChain.java:81)
   [junit4]   2> 	at org.apache.lucene.index.DefaultIndexingChain.startStoredFields(DefaultIndexingChain.java:258)
   [junit4]   2> 	at org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:295)
   [junit4]   2> 	at org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:234)
   [junit4]   2> 	at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:450)
   [junit4]   2> 	at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1477)
   [junit4]   2> 	... 2 more
   [junit4]   2> 
   [junit4]   2> dec 12, 2015 2:10:10 PM com.carrotsearch.randomizedtesting.RandomizedRunner$QueueUncaughtExceptionsHandler uncaughtException
   [junit4]   2> WARNING: Uncaught exception in thread: Thread[Thread-1644,5,TGRP-TestIndexWriterThreadsToSegments]
   [junit4]   2> java.lang.RuntimeException: java.nio.file.FileSystemException: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/test/J1/temp/lucene.index.TestIndexWriterThreadsToSegments_F72A04E52535E596-001/tempDir-001/_1ks.fdx: Too many open files
   [junit4]   2> 	at __randomizedtesting.SeedInfo.seed([F72A04E52535E596]:0)
   [junit4]   2> 	at org.apache.lucene.index.TestIndexWriterThreadsToSegments$2.run(TestIndexWriterThreadsToSegments.java:218)
   [junit4]   2> Caused by: java.nio.file.FileSystemException: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/test/J1/temp/lucene.index.TestIndexWriterThreadsToSegments_F72A04E52535E596-001/tempDir-001/_1ks.fdx: Too many open files
   [junit4]   2> 	at org.apache.lucene.mockfile.HandleLimitFS.onOpen(HandleLimitFS.java:49)
   [junit4]   2> 	at org.apache.lucene.mockfile.HandleTrackingFS.callOpenHook(HandleTrackingFS.java:82)
   [junit4]   2> 	at org.apache.lucene.mockfile.HandleTrackingFS.newOutputStream(HandleTrackingFS.java:161)
   [junit4]   2> 	at org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:198)
   [junit4]   2> 	at java.nio.file.Files.newOutputStream(Files.java:172)
   [junit4]   2> 	at org.apache.lucene.store.FSDirectory$FSIndexOutput.<init>(FSDirectory.java:276)
   [junit4]   2> 	at org.apache.lucene.store.FSDirectory.createOutput(FSDirectory.java:225)
   [junit4]   2> 	at org.apache.lucene.store.MockDirectoryWrapper.createOutput(MockDirectoryWrapper.java:593)
   [junit4]   2> 	at org.apache.lucene.store.LockValidatingDirectoryWrapper.createOutput(LockValidatingDirectoryWrapper.java:44)
   [junit4]   2> 	at org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:43)
   [junit4]   2> 	at org.apache.lucene.codecs.compressing.CompressingStoredFieldsWriter.<init>(CompressingStoredFieldsWriter.java:108)
   [junit4]   2> 	at org.apache.lucene.codecs.compressing.CompressingStoredFieldsFormat.fieldsWriter(CompressingStoredFieldsFormat.java:128)
   [junit4]   2> 	at org.apache.lucene.codecs.lucene50.Lucene50StoredFieldsFormat.fieldsWriter(Lucene50StoredFieldsFormat.java:183)
   [junit4]   2> 	at org.apache.lucene.codecs.asserting.AssertingStoredFieldsFormat.fieldsWriter(AssertingStoredFieldsFormat.java:49)
   [junit4]   2> 	at org.apache.lucene.index.DefaultIndexingChain.initStoredFieldsWriter(DefaultIndexingChain.java:81)
   [junit4]   2> 	at org.apache.lucene.index.DefaultIndexingChain.startStoredFields(DefaultIndexingChain.java:258)
   [junit4]   2> 	at org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:295)
   [junit4]   2> 	at org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:234)
   [junit4]   2> 	at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:450)
   [junit4]   2> 	at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1477)
   [junit4]   2> 	at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1256)
   [junit4]   2> 	at org.apache.lucene.index.TestIndexWriterThreadsToSegments$2.run(TestIndexWriterThreadsToSegments.java:207)
   [junit4]   2> 
   [junit4]   2> dec 12, 2015 2:10:10 PM com.carrotsearch.randomizedtesting.RandomizedRunner$QueueUncaughtExceptionsHandler uncaughtException
   [junit4]   2> WARNING: Uncaught exception in thread: Thread[Thread-1640,5,TGRP-TestIndexWriterThreadsToSegments]
   [junit4]   2> java.lang.RuntimeException: org.apache.lucene.store.AlreadyClosedException: this IndexWriter is closed
   [junit4]   2> 	at __randomizedtesting.SeedInfo.seed([F72A04E52535E596]:0)
   [junit4]   2> 	at org.apache.lucene.index.TestIndexWriterThreadsToSegments$2.run(TestIndexWriterThreadsToSegments.java:218)
   [junit4]   2> Caused by: org.apache.lucene.store.AlreadyClosedException: this IndexWriter is closed
   [junit4]   2> 	at org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:720)
   [junit4]   2> 	at org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:734)
   [junit4]   2> 	at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1473)
   [junit4]   2> 	at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1256)
   [junit4]   2> 	at org.apache.lucene.index.TestIndexWriterThreadsToSegments$2.run(TestIndexWriterThreadsToSegments.java:207)
   [junit4]   2> Caused by: java.nio.file.FileSystemException: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/test/J1/temp/lucene.index.TestIndexWriterThreadsToSegments_F72A04E52535E596-001/tempDir-001/_1ku.fdx: Too many open files
   [junit4]   2> 	at org.apache.lucene.mockfile.HandleLimitFS.onOpen(HandleLimitFS.java:49)
   [junit4]   2> 	at org.apache.lucene.mockfile.HandleTrackingFS.callOpenHook(HandleTrackingFS.java:82)
   [junit4]   2> 	at org.apache.lucene.mockfile.HandleTrackingFS.newOutputStream(HandleTrackingFS.java:161)
   [junit4]   2> 	at org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:198)
   [junit4]   2> 	at java.nio.file.Files.newOutputStream(Files.java:172)
   [junit4]   2> 	at org.apache.lucene.store.FSDirectory$FSIndexOutput.<init>(FSDirectory.java:276)
   [junit4]   2> 	at org.apache.lucene.store.FSDirectory.createOutput(FSDirectory.java:225)
   [junit4]   2> 	at org.apache.lucene.store.MockDirectoryWrapper.createOutput(MockDirectoryWrapper.java:593)
   [junit4]   2> 	at org.apache.lucene.store.LockValidatingDirectoryWrapper.createOutput(LockValidatingDirectoryWrapper.java:44)
   [junit4]   2> 	at org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:43)
   [junit4]   2> 	at org.apache.lucene.codecs.compressing.CompressingStoredFieldsWriter.<init>(CompressingStoredFieldsWriter.java:108)
   [junit4]   2> 	at org.apache.lucene.codecs.compressing.CompressingStoredFieldsFormat.fieldsWriter(CompressingStoredFieldsFormat.java:128)
   [junit4]   2> 	at org.apache.lucene.codecs.lucene50.Lucene50StoredFieldsFormat.fieldsWriter(Lucene50StoredFieldsFormat.java:183)
   [junit4]   2> 	at org.apache.lucene.codecs.asserting.AssertingStoredFieldsFormat.fieldsWriter(AssertingStoredFieldsFormat.java:49)
   [junit4]   2> 	at org.apache.lucene.index.DefaultIndexingChain.initStoredFieldsWriter(DefaultIndexingChain.java:81)
   [junit4]   2> 	at org.apache.lucene.index.DefaultIndexingChain.startStoredFields(DefaultIndexingChain.java:258)
   [junit4]   2> 	at org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:295)
   [junit4]   2> 	at org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:234)
   [junit4]   2> 	at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:450)
   [junit4]   2> 	at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1477)
   [junit4]   2> 	... 2 more
   [junit4]   2> 
   [junit4]   2> dec 12, 2015 2:10:10 PM com.carrotsearch.randomizedtesting.RandomizedRunner$QueueUncaughtExceptionsHandler uncaughtException
   [junit4]   2> WARNING: Uncaught exception in thread: Thread[Thread-1643,5,TGRP-TestIndexWriterThreadsToSegments]
   [junit4]   2> java.lang.RuntimeException: org.apache.lucene.store.AlreadyClosedException: this IndexWriter is closed
   [junit4]   2> 	at __randomizedtesting.SeedInfo.seed([F72A04E52535E596]:0)
   [junit4]   2> 	at org.apache.lucene.index.TestIndexWriterThreadsToSegments$2.run(TestIndexWriterThreadsToSegments.java:218)
   [junit4]   2> Caused by: org.apache.lucene.store.AlreadyClosedException: this IndexWriter is closed
   [junit4]   2> 	at org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:720)
   [junit4]   2> 	at org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:734)
   [junit4]   2> 	at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1473)
   [junit4]   2> 	at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1256)
   [junit4]   2> 	at org.apache.lucene.index.TestIndexWriterThreadsToSegments$2.run(TestIndexWriterThreadsToSegments.java:207)
   [junit4]   2> Caused by: java.nio.file.FileSystemException: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/test/J1/temp/lucene.index.TestIndexWriterThreadsToSegments_F72A04E52535E596-001/tempDir-001/_1ku.fdx: Too many open files
   [junit4]   2> 	at org.apache.lucene.mockfile.HandleLimitFS.onOpen(HandleLimitFS.java:49)
   [junit4]   2> 	at org.apache.lucene.mockfile.HandleTrackingFS.callOpenHook(HandleTrackingFS.java:82)
   [junit4]   2> 	at org.apache.lucene.mockfile.HandleTrackingFS.newOutputStream(HandleTrackingFS.java:161)
   [junit4]   2> 	at org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:198)
   [junit4]   2> 	at java.nio.file.Files.newOutputStream(Files.java:172)
   [junit4]   2> 	at org.apache.lucene.store.FSDirectory$FSIndexOutput.<init>(FSDirectory.java:276)
   [junit4]   2> 	at org.apache.lucene.store.FSDirectory.createOutput(FSDirectory.java:225)
   [junit4]   2> 	at org.apache.lucene.store.MockDirectoryWrapper.createOutput(MockDirectoryWrapper.java:593)
   [junit4]   2> 	at org.apache.lucene.store.LockValidatingDirectoryWrapper.createOutput(LockValidatingDirectoryWrapper.java:44)
   [junit4]   2> 	at org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:43)
   [junit4]   2> 	at org.apache.lucene.codecs.compressing.CompressingStoredFieldsWriter.<init>(CompressingStoredFieldsWriter.java:108)
   [junit4]   2> 	at org.apache.lucene.codecs.compressing.CompressingStoredFieldsFormat.fieldsWriter(CompressingStoredFieldsFormat.java:128)
   [junit4]   2> 	at org.apache.lucene.codecs.lucene50.Lucene50StoredFieldsFormat.fieldsWriter(Lucene50StoredFieldsFormat.java:183)
   [junit4]   2> 	at org.apache.lucene.codecs.asserting.AssertingStoredFieldsFormat.fieldsWriter(AssertingStoredFieldsFormat.java:49)
   [junit4]   2> 	at org.apache.lucene.index.DefaultIndexingChain.initStoredFieldsWriter(DefaultIndexingChain.java:81)
   [junit4]   2> 	at org.apache.lucene.index.DefaultIndexingChain.startStoredFields(DefaultIndexingChain.java:258)
   [junit4]   2> 	at org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:295)
   [junit4]   2> 	at org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:234)
   [junit4]   2> 	at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:450)
   [junit4]   2> 	at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1477)
   [junit4]   2> 	... 2 more
   [junit4]   2> 
   [junit4]   2> dec 12, 2015 2:10:10 PM com.carrotsearch.randomizedtesting.RandomizedRunner$QueueUncaughtExceptionsHandler uncaughtException
   [junit4]   2> WARNING: Uncaught exception in thread: Thread[Thread-1641,5,TGRP-TestIndexWriterThreadsToSegments]
   [junit4]   2> java.lang.RuntimeException: org.apache.lucene.store.AlreadyClosedException: this IndexWriter is closed
   [junit4]   2> 	at __randomizedtesting.SeedInfo.seed([F72A04E52535E596]:0)
   [junit4]   2> 	at org.apache.lucene.index.TestIndexWriterThreadsToSegments$2.run(TestIndexWriterThreadsToSegments.java:218)
   [junit4]   2> Caused by: org.apache.lucene.store.AlreadyClosedException: this IndexWriter is closed
   [junit4]   2> 	at org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:720)
   [junit4]   2> 	at org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:734)
   [junit4]   2> 	at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1473)
   [junit4]   2> 	at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1256)
   [junit4]   2> 	at org.apache.lucene.index.TestIndexWriterThreadsToSegments$2.run(TestIndexWriterThreadsToSegments.java:207)
   [junit4]   2> Caused by: java.nio.file.FileSystemException: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/test/J1/temp/lucene.index.TestIndexWriterThreadsToSegments_F72A04E52535E596-001/tempDir-001/_1ku.fdx: Too many open files
   [junit4]   2> 	at org.apache.lucene.mockfile.HandleLimitFS.onOpen(HandleLimitFS.java:49)
   [junit4]   2> 	at org.apache.lucene.mockfile.HandleTrackingFS.callOpenHook(HandleTrackingFS.java:82)
   [junit4]   2> 	at org.apache.lucene.mockfile.HandleTrackingFS.newOutputStream(HandleTrackingFS.java:161)
   [junit4]   2> 	at org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:198)
   [junit4]   2> 	at java.nio.file.Files.newOutputStream(Files.java:172)
   [junit4]   2> 	at org.apache.lucene.store.FSDirectory$FSIndexOutput.<init>(FSDirectory.java:276)
   [junit4]   2> 	at org.apache.lucene.store.FSDirectory.createOutput(FSDirectory.java:225)
   [junit4]   2> 	at org.apache.lucene.store.MockDirectoryWrapper.createOutput(MockDirectoryWrapper.java:593)
   [junit4]   2> 	at org.apache.lucene.store.LockValidatingDirectoryWrapper.createOutput(LockValidatingDirectoryWrapper.java:44)
   [junit4]   2> 	at org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:43)
   [junit4]   2> 	at org.apache.lucene.codecs.compressing.CompressingStoredFieldsWriter.<init>(CompressingStoredFieldsWriter.java:108)
   [junit4]   2> 	at org.apache.lucene.codecs.compressing.CompressingStoredFieldsFormat.fieldsWriter(CompressingStoredFieldsFormat.java:128)
   [junit4]   2> 	at org.apache.lucene.codecs.lucene50.Lucene50StoredFieldsFormat.fieldsWriter(Lucene50StoredFieldsFormat.java:183)
   [junit4]   2> 	at org.apache.lucene.codecs.asserting.AssertingStoredFieldsFormat.fieldsWriter(AssertingStoredFieldsFormat.java:49)
   [junit4]   2> 	at org.apache.lucene.index.DefaultIndexingChain.initStoredFieldsWriter(DefaultIndexingChain.java:81)
   [junit4]   2> 	at org.apache.lucene.index.DefaultIndexingChain.startStoredFields(DefaultIndexingChain.java:258)
   [junit4]   2> 	at org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:295)
   [junit4]   2> 	at org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:234)
   [junit4]   2> 	at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:450)
   [junit4]   2> 	at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1477)
   [junit4]   2> 	... 2 more
   [junit4]   2> 
   [junit4]   2> dec 12, 2015 2:10:10 PM com.carrotsearch.randomizedtesting.RandomizedRunner$QueueUncaughtExceptionsHandler uncaughtException
   [junit4]   2> WARNING: Uncaught exception in thread: Thread[Thread-1637,5,TGRP-TestIndexWriterThreadsToSegments]
   [junit4]   2> java.lang.RuntimeException: java.nio.file.FileSystemException: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/test/J1/temp/lucene.index.TestIndexWriterThreadsToSegments_F72A04E52535E596-001/tempDir-001/_1ku.fdx: Too many open files
   [junit4]   2> 	at __randomizedtesting.SeedInfo.seed([F72A04E52535E596]:0)
   [junit4]   2> 	at org.apache.lucene.index.TestIndexWriterThreadsToSegments$2.run(TestIndexWriterThreadsToSegments.java:218)
   [junit4]   2> Caused by: java.nio.file.FileSystemException: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/test/J1/temp/lucene.index.TestIndexWriterThreadsToSegments_F72A04E52535E596-001/tempDir-001/_1ku.fdx: Too many open files
   [junit4]   2> 	at org.apache.lucene.mockfile.HandleLimitFS.onOpen(HandleLimitFS.java:49)
   [junit4]   2> 	at org.apache.lucene.mockfile.HandleTrackingFS.callOpenHook(HandleTrackingFS.java:82)
   [junit4]   2> 	at org.apache.lucene.mockfile.HandleTrackingFS.newOutputStream(HandleTrackingFS.java:161)
   [junit4]   2> 	at org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:198)
   [junit4]   2> 	at java.nio.file.Files.newOutputStream(Files.java:172)
   [junit4]   2> 	at org.apache.lucene.store.FSDirectory$FSIndexOutput.<init>(FSDirectory.java:276)
   [junit4]   2> 	at org.apache.lucene.store.FSDirectory.createOutput(FSDirectory.java:225)
   [junit4]   2> 	at org.apache.lucene.store.MockDirectoryWrapper.createOutput(MockDirectoryWrapper.java:593)
   [junit4]   2> 	at org.apache.lucene.store.LockValidatingDirectoryWrapper.createOutput(LockValidatingDirectoryWrapper.java:44)
   [junit4]   2> 	at org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:43)
   [junit4]   2> 	at org.apache.lucene.codecs.compressing.CompressingStoredFieldsWriter.<init>(CompressingStoredFieldsWriter.java:108)
   [junit4]   2> 	at org.apache.lucene.codecs.compressing.CompressingStoredFieldsFormat.fieldsWriter(CompressingStoredFieldsFormat.java:128)
   [junit4]   2> 	at org.apache.lucene.codecs.lucene50.Lucene50StoredFieldsFormat.fieldsWriter(Lucene50StoredFieldsFormat.java:183)
   [junit4]   2> 	at org.apache.lucene.codecs.asserting.AssertingStoredFieldsFormat.fieldsWriter(AssertingStoredFieldsFormat.java:49)
   [junit4]   2> 	at org.apache.lucene.index.DefaultIndexingChain.initStoredFieldsWriter(DefaultIndexingChain.java:81)
   [junit4]   2> 	at org.apache.lucene.index.DefaultIndexingChain.startStoredFields(DefaultIndexingChain.java:258)
   [junit4]   2> 	at org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:295)
   [junit4]   2> 	at org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:234)
   [junit4]   2> 	at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:450)
   [junit4]   2> 	at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1477)
   [junit4]   2> 	at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1256)
   [junit4]   2> 	at org.apache.lucene.index.TestIndexWriterThreadsToSegments$2.run(TestIndexWriterThreadsToSegments.java:207)
   [junit4]   2> 
   [junit4]   2> dec 12, 2015 2:10:10 PM com.carrotsearch.randomizedtesting.RandomizedRunner$QueueUncaughtExceptionsHandler uncaughtException
   [junit4]   2> WARNING: Uncaught exception in thread: Thread[Thread-1645,5,TGRP-TestIndexWriterThreadsToSegments]
   [junit4]   2> java.lang.RuntimeException: java.nio.file.FileSystemException: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/test/J1/temp/lucene.index.TestIndexWriterThreadsToSegments_F72A04E52535E596-001/tempDir-001/_1kp.fdx: Too many open files
   [junit4]   2> 	at __randomizedtesting.SeedInfo.seed([F72A04E52535E596]:0)
   [junit4]   2> 	at org.apache.lucene.index.TestIndexWriterThreadsToSegments$2.run(TestIndexWriterThreadsToSegments.java:218)
   [junit4]   2> Caused by: java.nio.file.FileSystemException: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/test/J1/temp/lucene.index.TestIndexWriterThreadsToSegments_F72A04E52535E596-001/tempDir-001/_1kp.fdx: Too many open files
   [junit4]   2> 	at org.apache.lucene.mockfile.HandleLimitFS.onOpen(HandleLimitFS.java:49)
   [junit4]   2> 	at org.apache.lucene.mockfile.HandleTrackingFS.callOpenHook(HandleTrackingFS.java:82)
   [junit4]   2> 	at org.apache.lucene.mockfile.HandleTrackingFS.newOutputStream(HandleTrackingFS.java:161)
   [junit4]   2> 	at org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:198)
   [junit4]   2> 	at java.nio.file.Files.newOutputStream(Files.java:172)
   [junit4]   2> 	at org.apache.lucene.store.FSDirectory$FSIndexOutput.<init>(FSDirectory.java:276)
   [junit4]   2> 	at org.apache.lucene.store.FSDirectory.createOutput(FSDirectory.java:225)
   [junit4]   2> 	at org.apache.lucene.store.MockDirectoryWrapper.createOutput(MockDirectoryWrapper.java:593)
   [junit4]   2> 	at org.apache.lucene.store.LockValidatingDirectoryWrapper.createOutput(LockValidatingDirectoryWrapper.java:44)
   [junit4]   2> 	at org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:43)
   [junit4]   2> 	at org.apache.lucene.codecs.compressing.CompressingStoredFieldsWriter.<init>(CompressingStoredFieldsWriter.java:108)
   [junit4]   2> 	at org.apache.lucene.codecs.compressing.CompressingStoredFieldsFormat.fieldsWriter(CompressingStoredFieldsFormat.java:128)
   [junit4]   2> 	at org.apache.lucene.codecs.lucene50.Lucene50StoredFieldsFormat.fieldsWriter(Lucene50StoredFieldsFormat.java:183)
   [junit4]   2> 	at org.apache.lucene.codecs.asserting.AssertingStoredFieldsFormat.fieldsWriter(AssertingStoredFieldsFormat.java:49)
   [junit4]   2> 	at org.apache.lucene.index.DefaultIndexingChain.initStoredFieldsWriter(DefaultIndexingChain.java:81)
   [junit4]   2> 	at org.apache.lucene.index.DefaultIndexingChain.startStoredFields(DefaultIndexingChain.java:258)
   [junit4]   2> 	at org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:295)
   [junit4]   2> 	at org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:234)
   [junit4]   2> 	at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:450)
   [junit4]   2> 	at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1477)
   [junit4]   2> 	at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1256)
   [junit4]   2> 	at org.apache.lucene.index.TestIndexWriterThreadsToSegments$2.run(TestIndexWriterThreadsToSegments.java:207)
   [junit4]   2> 
   [junit4]   2> dec 12, 2015 2:10:10 PM com.carrotsearch.randomizedtesting.RandomizedRunner$QueueUncaughtExceptionsHandler uncaughtException
   [junit4]   2> WARNING: Uncaught exception in thread: Thread[Thread-1646,5,TGRP-TestIndexWriterThreadsToSegments]
   [junit4]   2> java.lang.RuntimeException: java.nio.file.FileSystemException: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/test/J1/temp/lucene.index.TestIndexWriterThreadsToSegments_F72A04E52535E596-001/tempDir-001/_1kt.fdt: Too many open files
   [junit4]   2> 	at __randomizedtesting.SeedInfo.seed([F72A04E52535E596]:0)
   [junit4]   2> 	at org.apache.lucene.index.TestIndexWriterThreadsToSegments$2.run(TestIndexWriterThreadsToSegments.java:218)
   [junit4]   2> Caused by: java.nio.file.FileSystemException: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/test/J1/temp/lucene.index.TestIndexWriterThreadsToSegments_F72A04E52535E596-001/tempDir-001/_1kt.fdt: Too many open files
   [junit4]   2> 	at org.apache.lucene.mockfile.HandleLimitFS.onOpen(HandleLimitFS.java:49)
   [junit4]   2> 	at org.apache.lucene.mockfile.HandleTrackingFS.callOpenHook(HandleTrackingFS.java:82)
   [junit4]   2> 	at org.apache.lucene.mockfile.HandleTrackingFS.newOutputStream(HandleTrackingFS.java:161)
   [junit4]   2> 	at org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:198)
   [junit4]   2> 	at java.nio.file.Files.newOutputStream(Files.java:172)
   [junit4]   2> 	at org.apache.lucene.store.FSDirectory$FSIndexOutput.<init>(FSDirectory.java:276)
   [junit4]   2> 	at org.apache.lucene.store.FSDirectory.createOutput(FSDirectory.java:225)
   [junit4]   2> 	at org.apache.lucene.store.MockDirectoryWrapper.createOutput(MockDirectoryWrapper.java:593)
   [junit4]   2> 	at org.apache.lucene.store.LockValidatingDirectoryWrapper.createOutput(LockValidatingDirectoryWrapper.java:44)
   [junit4]   2> 	at org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:43)
   [junit4]   2> 	at org.apache.lucene.codecs.compressing.CompressingStoredFieldsWriter.<init>(CompressingStoredFieldsWriter.java:111)
   [junit4]   2> 	at org.apache.lucene.codecs.compressing.CompressingStoredFieldsFormat.fieldsWriter(CompressingStoredFieldsFormat.java:128)
   [junit4]   2> 	at org.apache.lucene.codecs.lucene50.Lucene50StoredFieldsFormat.fieldsWriter(Lucene50StoredFieldsFormat.java:183)
   [junit4]   2> 	at org.apache.lucene.codecs.asserting.AssertingStoredFieldsFormat.fieldsWriter(AssertingStoredFieldsFormat.java:49)
   [junit4]   2> 	at org.apache.lucene.index.DefaultIndexingChain.initStoredFieldsWriter(DefaultIndexingChain.java:81)
   [junit4]   2> 	at org.apache.lucene.index.DefaultIndexingChain.startStoredFields(DefaultIndexingChain.java:258)
   [junit4]   2> 	at org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:295)
   [junit4]   2> 	at org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:234)
   [junit4]   2> 	at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:450)
   [junit4]   2> 	at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1477)
   [junit4]   2> 	at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1256)
   [junit4]   2> 	at org.apache.lucene.index.TestIndexWriterThreadsToSegments$2.run(TestIndexWriterThreadsToSegments.java:207)
   [junit4]   2> 
   [junit4]   2> dec 12, 2015 4:08:42 PM com.carrotsearch.randomizedtesting.ThreadLeakControl$2 evaluate
   [junit4]   2> WARNING: Suite execution timed out: org.apache.lucene.index.TestIndexWriterThreadsToSegments
   [junit4]   2>    1) Thread[id=1, name=main, state=WAITING, group=main]
   [junit4]   2>         at java.lang.Object.wait(Native Method)
   [junit4]   2>         at java.lang.Thread.join(Thread.java:1281)
   [junit4]   2>         at java.lang.Thread.join(Thread.java:1355)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:601)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner.run(RandomizedRunner.java:450)
   [junit4]   2>         at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.execute(SlaveMain.java:243)
   [junit4]   2>         at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.main(SlaveMain.java:354)
   [junit4]   2>         at com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe.main(SlaveMainSafe.java:10)
   [junit4]   2>    2) Thread[id=1716, name=Thread-1639, state=WAITING, group=TGRP-TestIndexWriterThreadsToSegments]
   [junit4]   2>         at sun.misc.Unsafe.park(Native Method)
   [junit4]   2>         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4]   2>         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
   [junit4]   2>         at java.util.concurrent.CyclicBarrier.dowait(CyclicBarrier.java:227)
   [junit4]   2>         at java.util.concurrent.CyclicBarrier.await(CyclicBarrier.java:355)
   [junit4]   2>         at org.apache.lucene.index.TestIndexWriterThreadsToSegments$2.run(TestIndexWriterThreadsToSegments.java:215)
   [junit4]   2>    3) Thread[id=1698, name=SUITE-TestIndexWriterThreadsToSegments-seed#[F72A04E52535E596], state=RUNNABLE, group=TGRP-TestIndexWriterThreadsToSegments]
   [junit4]   2>         at java.lang.Thread.getStackTrace(Thread.java:1589)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$4.run(ThreadLeakControl.java:688)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$4.run(ThreadLeakControl.java:685)
   [junit4]   2>         at java.security.AccessController.doPrivileged(Native Method)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl.getStackTrace(ThreadLeakControl.java:685)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl.getThreadsWithTraces(ThreadLeakControl.java:701)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl.formatThreadStacksFull(ThreadLeakControl.java:681)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl.access$1000(ThreadLeakControl.java:64)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$2.evaluate(ThreadLeakControl.java:414)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:681)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner.access$200(RandomizedRunner.java:140)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$2.run(RandomizedRunner.java:591)
   [junit4]   2>    4) Thread[id=8, name=JUnit4-serializer-daemon, state=TIMED_WAITING, group=main]
   [junit4]   2>         at java.lang.Thread.sleep(Native Method)
   [junit4]   2>         at com.carrotsearch.ant.tasks.junit4.events.Serializer$1.run(Serializer.java:47)
   [junit4]   2>    5) Thread[id=1715, name=Thread-1638, state=WAITING, group=TGRP-TestIndexWriterThreadsToSegments]
   [junit4]   2>         at sun.misc.Unsafe.park(Native Method)
   [junit4]   2>         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4]   2>         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
   [junit4]   2>         at java.util.concurrent.CyclicBarrier.dowait(CyclicBarrier.java:227)
   [junit4]   2>         at java.util.concurrent.CyclicBarrier.await(CyclicBarrier.java:355)
   [junit4]   2>         at org.apache.lucene.index.TestIndexWriterThreadsToSegments$2.run(TestIndexWriterThreadsToSegments.java:215)
   [junit4]   2>    6) Thread[id=1699, name=TEST-TestIndexWriterThreadsToSegments.testSegmentCountOnFlushRandom-seed#[F72A04E52535E596], state=WAITING, group=TGRP-TestIndexWriterThreadsToSegments]
   [junit4]   2>         at java.lang.Object.wait(Native Method)
   [junit4]   2>         at java.lang.Thread.join(Thread.java:1281)
   [junit4]   2>         at java.lang.Thread.join(Thread.java:1355)
   [junit4]   2>         at org.apache.lucene.index.TestIndexWriterThreadsToSegments.testSegmentCountOnFlushRandom(TestIndexWriterThreadsToSegments.java:226)
   [junit4]   2>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   2>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   [junit4]   2>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2>         at java.lang.reflect.Method.invoke(Method.java:606)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
   [junit4]   2>         at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
   [junit4]   2>         at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2>         at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
   [junit4]   2>         at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2>         at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
   [junit4]   2>         at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   2>         at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2>         at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2>         at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
   [junit4]   2>         at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestIndexWriterThreadsToSegments -Dtests.method=testSegmentCountOnFlushRandom -Dtests.seed=F72A04E52535E596 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=sr_RS_#Latn -Dtests.timezone=America/Santa_Isabel -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR   7199s J1 | TestIndexWriterThreadsToSegments.testSegmentCountOnFlushRandom <<<
   [junit4]    > Throwable #1: java.lang.Exception: Test abandoned because suite timeout was reached.
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([F72A04E52535E596]:0)
   [junit4]   2> dec 12, 2015 4:08:42 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 3 leaked thread(s).
   [junit4]   2> dec 12, 2015 4:09:02 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> SEVERE: 3 threads leaked from SUITE scope at org.apache.lucene.index.TestIndexWriterThreadsToSegments: 
   [junit4]   2>    1) Thread[id=1715, name=Thread-1638, state=WAITING, group=TGRP-TestIndexWriterThreadsToSegments]
   [junit4]   2>         at sun.misc.Unsafe.park(Native Method)
   [junit4]   2>         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4]   2>         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
   [junit4]   2>         at java.util.concurrent.CyclicBarrier.dowait(CyclicBarrier.java:227)
   [junit4]   2>         at java.util.concurrent.CyclicBarrier.await(CyclicBarrier.java:355)
   [junit4]   2>         at org.apache.lucene.index.TestIndexWriterThreadsToSegments$2.run(TestIndexWriterThreadsToSegments.java:215)
   [junit4]   2>    2) Thread[id=1699, name=TEST-TestIndexWriterThreadsToSegments.testSegmentCountOnFlushRandom-seed#[F72A04E52535E596], state=WAITING, group=TGRP-TestIndexWriterThreadsToSegments]
   [junit4]   2>         at java.lang.Object.wait(Native Method)
   [junit4]   2>         at java.lang.Thread.join(Thread.java:1281)
   [junit4]   2>         at java.lang.Thread.join(Thread.java:1355)
   [junit4]   2>         at org.apache.lucene.index.TestIndexWriterThreadsToSegments.testSegmentCountOnFlushRandom(TestIndexWriterThreadsToSegments.java:226)
   [junit4]   2>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   2>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   [junit4]   2>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2>         at java.lang.reflect.Method.invoke(Method.java:606)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
   [junit4]   2>         at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
   [junit4]   2>         at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2>         at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
   [junit4]   2>         at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2>         at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
   [junit4]   2>         at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   2>         at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2>         at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2>         at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
   [junit4]   2>         at java.lang.Thread.run(Thread.java:745)
   [junit4]   2>    3) Thread[id=1716, name=Thread-1639, state=WAITING, group=TGRP-TestIndexWriterThreadsToSegments]
   [junit4]   2>         at sun.misc.Unsafe.park(Native Method)
   [junit4]   2>         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4]   2>         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
   [junit4]   2>         at java.util.concurrent.CyclicBarrier.dowait(CyclicBarrier.java:227)
   [junit4]   2>         at java.util.concurrent.CyclicBarrier.await(CyclicBarrier.java:355)
   [junit4]   2>         at org.apache.lucene.index.TestIndexWriterThreadsToSegments$2.run(TestIndexWriterThreadsToSegments.java:215)
   [junit4]   2> dec 12, 2015 4:09:02 PM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
   [junit4]   2> INFO: Starting to interrupt leaked threads:
   [junit4]   2>    1) Thread[id=1715, name=Thread-1638, state=WAITING, group=TGRP-TestIndexWriterThreadsToSegments]
   [junit4]   2>    2) Thread[id=1699, name=TEST-TestIndexWriterThreadsToSegments.testSegmentCountOnFlushRandom-seed#[F72A04E52535E596], state=WAITING, group=TGRP-TestIndexWriterThreadsToSegments]
   [junit4]   2>    3) Thread[id=1716, name=Thread-1639, state=WAITING, group=TGRP-TestIndexWriterThreadsToSegments]
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/test/J1/temp/lucene.index.TestIndexWriterThreadsToSegments_F72A04E52535E596-001
   [junit4]   2> Dec 13, 2015 12:09:02 AM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
   [junit4]   2> INFO: All leaked threads terminated.
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene54): {field=Lucene50(blocksize=128)}, docValues:{}, sim=DefaultSimilarity, locale=sr_RS_#Latn, timezone=America/Santa_Isabel
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_80 (64-bit)/cpus=4,threads=1,free=130871024,total=402128896
   [junit4]   2> NOTE: All tests run in this JVM: [TestTermScorer, TestDocCount, TestSortedSetSortField, TestIndexWriterLockRelease, TestIndexWriterExceptions, TestBoostQuery, TestDisjunctionMaxQuery, TestIndexWriterDeleteByQuery, Test2BNumericDocValues, TestSegmentMerger, TestDemoParallelLeafReader, TestFlex, TestFastDecompressionMode, TestBoolean2, TestControlledRealTimeReopenThread, TestTermVectorsReader, TestRecyclingIntBlockAllocator, TestBlendedTermQuery, TestFSTs, TestFieldValueFilter, TestIndexWriterThreadsToSegments]
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestIndexWriterThreadsToSegments -Dtests.seed=F72A04E52535E596 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=sr_RS_#Latn -Dtests.timezone=America/Santa_Isabel -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR   0.00s J1 | TestIndexWriterThreadsToSegments (suite) <<<
   [junit4]    > Throwable #1: java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([F72A04E52535E596]:0)Throwable #2: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=1721, name=Thread-1644, state=RUNNABLE, group=TGRP-TestIndexWriterThreadsToSegments]
   [junit4]    > Caused by: java.lang.RuntimeException: java.nio.file.FileSystemException: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/test/J1/temp/lucene.index.TestIndexWriterThreadsToSegments_F72A04E52535E596-001/tempDir-001/_1ks.fdx: Too many open files
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([F72A04E52535E596]:0)
   [junit4]    > 	at org.apache.lucene.index.TestIndexWriterThreadsToSegments$2.run(TestIndexWriterThreadsToSegments.java:218)
   [junit4]    > Caused by: java.nio.file.FileSystemException: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/test/J1/temp/lucene.index.TestIndexWriterThreadsToSegments_F72A04E52535E596-001/tempDir-001/_1ks.fdx: Too many open files
   [junit4]    > 	at org.apache.lucene.mockfile.HandleLimitFS.onOpen(HandleLimitFS.java:49)
   [junit4]    > 	at org.apache.lucene.mockfile.HandleTrackingFS.callOpenHook(HandleTrackingFS.java:82)
   [junit4]    > 	at org.apache.lucene.mockfile.HandleTrackingFS.newOutputStream(HandleTrackingFS.java:161)
   [junit4]    > 	at org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:198)
   [junit4]    > 	at java.nio.file.Files.newOutputStream(Files.java:172)
   [junit4]    > 	at org.apache.lucene.store.FSDirectory$FSIndexOutput.<init>(FSDirectory.java:276)
   [junit4]    > 	at org.apache.lucene.store.FSDirectory.createOutput(FSDirectory.java:225)
   [junit4]    > 	at org.apache.lucene.store.MockDirectoryWrapper.createOutput(MockDirectoryWrapper.java:593)
   [junit4]    > 	at org.apache.lucene.store.LockValidatingDirectoryWrapper.createOutput(LockValidatingDirectoryWrapper.java:44)
   [junit4]    > 	at org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:43)
   [junit4]    > 	at org.apache.lucene.codecs.compressing.CompressingStoredFieldsWriter.<init>(CompressingStoredFieldsWriter.java:108)
   [junit4]    > 	at org.apache.lucene.codecs.compressing.CompressingStoredFieldsFormat.fieldsWriter(CompressingStoredFieldsFormat.java:128)
   [junit4]    > 	at org.apache.lucene.codecs.lucene50.Lucene50StoredFieldsFormat.fieldsWriter(Lucene50StoredFieldsFormat.java:183)
   [junit4]    > 	at org.apache.lucene.codecs.asserting.AssertingStoredFieldsFormat.fieldsWriter(AssertingStoredFieldsFormat.java:49)
   [junit4]    > 	at org.apache.lucene.index.DefaultIndexingChain.initStoredFieldsWriter(DefaultIndexingChain.java:81)
   [junit4]    > 	at org.apache.lucene.index.DefaultIndexingChain.startStoredFields(DefaultIndexingChain.java:258)
   [junit4]    > 	at org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:295)
   [junit4]    > 	at org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:234)
   [junit4]    > 	at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:450)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1477)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1256)
   [junit4]    > 	at org.apache.lucene.index.TestIndexWriterThreadsToSegments$2.run(TestIndexWriterThreadsToSegments.java:207)Throwable #3: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=1719, name=Thread-1642, state=RUNNABLE, group=TGRP-TestIndexWriterThreadsToSegments]
   [junit4]    > Caused by: java.lang.RuntimeException: org.apache.lucene.store.AlreadyClosedException: this IndexWriter is closed
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([F72A04E52535E596]:0)
   [junit4]    > 	at org.apache.lucene.index.TestIndexWriterThreadsToSegments$2.run(TestIndexWriterThreadsToSegments.java:218)
   [junit4]    > Caused by: org.apache.lucene.store.AlreadyClosedException: this IndexWriter is closed
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:720)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:734)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1473)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1256)
   [junit4]    > 	at org.apache.lucene.index.TestIndexWriterThreadsToSegments$2.run(TestIndexWriterThreadsToSegments.java:207)
   [junit4]    > Caused by: java.nio.file.FileSystemException: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/test/J1/temp/lucene.index.TestIndexWriterThreadsToSegments_F72A04E52535E596-001/tempDir-001/_1ku.fdx: Too many open files
   [junit4]    > 	at org.apache.lucene.mockfile.HandleLimitFS.onOpen(HandleLimitFS.java:49)
   [junit4]    > 	at org.apache.lucene.mockfile.HandleTrackingFS.callOpenHook(HandleTrackingFS.java:82)
   [junit4]    > 	at org.apache.lucene.mockfile.HandleTrackingFS.newOutputStream(HandleTrackingFS.java:161)
   [junit4]    > 	at org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:198)
   [junit4]    > 	at java.nio.file.Files.newOutputStream(Files.java:172)
   [junit4]    > 	at org.apache.lucene.store.FSDirectory$FSIndexOutput.<init>(FSDirectory.java:276)
   [junit4]    > 	at org.apache.lucene.store.FSDirectory.createOutput(FSDirectory.java:225)
   [junit4]    > 	at org.apache.lucene.store.MockDirectoryWrapper.createOutput(MockDirectoryWrapper.java:593)
   [junit4]    > 	at org.apache.lucene.store.LockValidatingDirectoryWrapper.createOutput(LockValidatingDirectoryWrapper.java:44)
   [junit4]    > 	at org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:43)
   [junit4]    > 	at org.apache.lucene.codecs.compressing.CompressingStoredFieldsWriter.<init>(CompressingStoredFieldsWriter.java:108)
   [junit4]    > 	at org.apache.lucene.codecs.compressing.CompressingStoredFieldsFormat.fieldsWriter(CompressingStoredFieldsFormat.java:128)
   [junit4]    > 	at org.apache.lucene.codecs.lucene50.Lucene50StoredFieldsFormat.fieldsWriter(Lucene50StoredFieldsFormat.java:183)
   [junit4]    > 	at org.apache.lucene.codecs.asserting.AssertingStoredFieldsFormat.fieldsWriter(AssertingStoredFieldsFormat.java:49)
   [junit4]    > 	at org.apache.lucene.index.DefaultIndexingChain.initStoredFieldsWriter(DefaultIndexingChain.java:81)
   [junit4]    > 	at org.apache.lucene.index.DefaultIndexingChain.startStoredFields(DefaultIndexingChain.java:258)
   [junit4]    > 	at org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:295)
   [junit4]    > 	at org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:234)
   [junit4]    > 	at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:450)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1477)
   [junit4]    > 	... 2 moreThrowable #4: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=1722, name=Thread-1645, state=RUNNABLE, group=TGRP-TestIndexWriterThreadsToSegments]
   [junit4]    > Caused by: java.lang.RuntimeException: java.nio.file.FileSystemException: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/test/J1/temp/lucene.index.TestIndexWriterThreadsToSegments_F72A04E52535E596-001/tempDir-001/_1kp.fdx: Too many open files
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([F72A04E52535E596]:0)
   [junit4]    > 	at org.apache.lucene.index.TestIndexWriterThreadsToSegments$2.run(TestIndexWriterThreadsToSegments.java:218)
   [junit4]    > Caused by: java.nio.file.FileSystemException: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/test/J1/temp/lucene.index.TestIndexWriterThreadsToSegments_F72A04E52535E596-001/tempDir-001/_1kp.fdx: Too many open files
   [junit4]    > 	at org.apache.lucene.mockfile.HandleLimitFS.onOpen(HandleLimitFS.java:49)
   [junit4]    > 	at org.apache.lucene.mockfile.HandleTrackingFS.callOpenHook(HandleTrackingFS.java:82)
   [junit4]    > 	at org.apache.lucene.mockfile.HandleTrackingFS.newOutputStream(HandleTrackingFS.java:161)
   [junit4]    > 	at org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:198)
   [junit4]    > 	at java.nio.file.Files.newOutputStream(Files.java:172)
   [junit4]    > 	at org.apache.lucene.store.FSDirectory$FSIndexOutput.<init>(FSDirectory.java:276)
   [junit4]    > 	at org.apache.lucene.store.FSDirectory.createOutput(FSDirectory.java:225)
   [junit4]    > 	at org.apache.lucene.store.MockDirectoryWrapper.createOutput(MockDirectoryWrapper.java:593)
   [junit4]    > 	at org.apache.lucene.store.LockValidatingDirectoryWrapper.createOutput(LockValidatingDirectoryWrapper.java:44)
   [junit4]    > 	at org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:43)
   [junit4]    > 	at org.apache.lucene.codecs.compressing.CompressingStoredFieldsWriter.<init>(CompressingStoredFieldsWriter.java:108)
   [junit4]    > 	at org.apache.lucene.codecs.compressing.CompressingStoredFieldsFormat.fieldsWriter(CompressingStoredFieldsFormat.java:128)
   [junit4]    > 	at org.apache.lucene.codecs.lucene50.Lucene50StoredFieldsFormat.fieldsWriter(Lucene50StoredFieldsFormat.java:183)
   [junit4]    > 	at org.apache.lucene.codecs.asserting.AssertingStoredFieldsFormat.fieldsWriter(AssertingStoredFieldsFormat.java:49)
   [junit4]    > 	at org.apache.lucene.index.DefaultIndexingChain.initStoredFieldsWriter(DefaultIndexingChain.java:81)
   [junit4]    > 	at org.apache.lucene.index.DefaultIndexingChain.startStoredFields(DefaultIndexingChain.java:258)
   [junit4]    > 	at org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:295)
   [junit4]    > 	at org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:234)
   [junit4]    > 	at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:450)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1477)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1256)
   [junit4]    > 	at org.apache.lucene.index.TestIndexWriterThreadsToSegments$2.run(TestIndexWriterThreadsToSegments.java:207)Throwable #5: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=1723, name=Thread-1646, state=RUNNABLE, group=TGRP-TestIndexWriterThreadsToSegments]
   [junit4]    > Caused by: java.lang.RuntimeException: java.nio.file.FileSystemException: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/test/J1/temp/lucene.index.TestIndexWriterThreadsToSegments_F72A04E52535E596-001/tempDir-001/_1kt.fdt: Too many open files
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([F72A04E52535E596]:0)
   [junit4]    > 	at org.apache.lucene.index.TestIndexWriterThreadsToSegments$2.run(TestIndexWriterThreadsToSegments.java:218)
   [junit4]    > Caused by: java.nio.file.FileSystemException: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/test/J1/temp/lucene.index.TestIndexWriterThreadsToSegments_F72A04E52535E596-001/tempDir-001/_1kt.fdt: Too many open files
   [junit4]    > 	at org.apache.lucene.mockfile.HandleLimitFS.onOpen(HandleLimitFS.java:49)
   [junit4]    > 	at org.apache.lucene.mockfile.HandleTrackingFS.callOpenHook(HandleTrackingFS.java:82)
   [junit4]    > 	at org.apache.lucene.mockfile.HandleTrackingFS.newOutputStream(HandleTrackingFS.java:161)
   [junit4]    > 	at org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:198)
   [junit4]    > 	at java.nio.file.Files.newOutputStream(Files.java:172)
   [junit4]    > 	at org.apache.lucene.store.FSDirectory$FSIndexOutput.<init>(FSDirectory.java:276)
   [junit4]    > 	at org.apache.lucene.store.FSDirectory.createOutput(FSDirectory.java:225)
   [junit4]    > 	at org.apache.lucene.store.MockDirectoryWrapper.createOutput(MockDirectoryWrapper.java:593)
   [junit4]    > 	at org.apache.lucene.store.LockValidatingDirectoryWrapper.createOutput(LockValidatingDirectoryWrapper.java:44)
   [junit4]    > 	at org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:43)
   [junit4]    > 	at org.apache.lucene.codecs.compressing.CompressingStoredFieldsWriter.<init>(CompressingStoredFieldsWriter.java:111)
   [junit4]    > 	at org.apache.lucene.codecs.compressing.CompressingStoredFieldsFormat.fieldsWriter(CompressingStoredFieldsFormat.java:128)
   [junit4]    > 	at org.apache.lucene.codecs.lucene50.Lucene50StoredFieldsFormat.fieldsWriter(Lucene50StoredFieldsFormat.java:183)
   [junit4]    > 	at org.apache.lucene.codecs.asserting.AssertingStoredFieldsFormat.fieldsWriter(AssertingStoredFieldsFormat.java:49)
   [junit4]    > 	at org.apache.lucene.index.DefaultIndexingChain.initStoredFieldsWriter(DefaultIndexingChain.java:81)
   [junit4]    > 	at org.apache.lucene.index.DefaultIndexingChain.startStoredFields(DefaultIndexingChain.java:258)
   [junit4]    > 	at org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:295)
   [junit4]    > 	at org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:234)
   [junit4]    > 	at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:450)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1477)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1256)
   [junit4]    > 	at org.apache.lucene.index.TestIndexWriterThreadsToSegments$2.run(TestIndexWriterThreadsToSegments.java:207)Throwable #6: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thr

[...truncated too long message...]

> 	at org.apache.lucene.store.MockDirectoryWrapper.createOutput(MockDirectoryWrapper.java:593)
   [junit4]    > 	at org.apache.lucene.store.LockValidatingDirectoryWrapper.createOutput(LockValidatingDirectoryWrapper.java:44)
   [junit4]    > 	at org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:43)
   [junit4]    > 	at org.apache.lucene.codecs.compressing.CompressingStoredFieldsWriter.<init>(CompressingStoredFieldsWriter.java:108)
   [junit4]    > 	at org.apache.lucene.codecs.compressing.CompressingStoredFieldsFormat.fieldsWriter(CompressingStoredFieldsFormat.java:128)
   [junit4]    > 	at org.apache.lucene.codecs.lucene50.Lucene50StoredFieldsFormat.fieldsWriter(Lucene50StoredFieldsFormat.java:183)
   [junit4]    > 	at org.apache.lucene.codecs.asserting.AssertingStoredFieldsFormat.fieldsWriter(AssertingStoredFieldsFormat.java:49)
   [junit4]    > 	at org.apache.lucene.index.DefaultIndexingChain.initStoredFieldsWriter(DefaultIndexingChain.java:81)
   [junit4]    > 	at org.apache.lucene.index.DefaultIndexingChain.startStoredFields(DefaultIndexingChain.java:258)
   [junit4]    > 	at org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:295)
   [junit4]    > 	at org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:234)
   [junit4]    > 	at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:450)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1477)
   [junit4]    > 	... 2 moreThrowable #8: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=1718, name=Thread-1641, state=RUNNABLE, group=TGRP-TestIndexWriterThreadsToSegments]
   [junit4]    > Caused by: java.lang.RuntimeException: org.apache.lucene.store.AlreadyClosedException: this IndexWriter is closed
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([F72A04E52535E596]:0)
   [junit4]    > 	at org.apache.lucene.index.TestIndexWriterThreadsToSegments$2.run(TestIndexWriterThreadsToSegments.java:218)
   [junit4]    > Caused by: org.apache.lucene.store.AlreadyClosedException: this IndexWriter is closed
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:720)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:734)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1473)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1256)
   [junit4]    > 	at org.apache.lucene.index.TestIndexWriterThreadsToSegments$2.run(TestIndexWriterThreadsToSegments.java:207)
   [junit4]    > Caused by: java.nio.file.FileSystemException: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/test/J1/temp/lucene.index.TestIndexWriterThreadsToSegments_F72A04E52535E596-001/tempDir-001/_1ku.fdx: Too many open files
   [junit4]    > 	at org.apache.lucene.mockfile.HandleLimitFS.onOpen(HandleLimitFS.java:49)
   [junit4]    > 	at org.apache.lucene.mockfile.HandleTrackingFS.callOpenHook(HandleTrackingFS.java:82)
   [junit4]    > 	at org.apache.lucene.mockfile.HandleTrackingFS.newOutputStream(HandleTrackingFS.java:161)
   [junit4]    > 	at org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:198)
   [junit4]    > 	at java.nio.file.Files.newOutputStream(Files.java:172)
   [junit4]    > 	at org.apache.lucene.store.FSDirectory$FSIndexOutput.<init>(FSDirectory.java:276)
   [junit4]    > 	at org.apache.lucene.store.FSDirectory.createOutput(FSDirectory.java:225)
   [junit4]    > 	at org.apache.lucene.store.MockDirectoryWrapper.createOutput(MockDirectoryWrapper.java:593)
   [junit4]    > 	at org.apache.lucene.store.LockValidatingDirectoryWrapper.createOutput(LockValidatingDirectoryWrapper.java:44)
   [junit4]    > 	at org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:43)
   [junit4]    > 	at org.apache.lucene.codecs.compressing.CompressingStoredFieldsWriter.<init>(CompressingStoredFieldsWriter.java:108)
   [junit4]    > 	at org.apache.lucene.codecs.compressing.CompressingStoredFieldsFormat.fieldsWriter(CompressingStoredFieldsFormat.java:128)
   [junit4]    > 	at org.apache.lucene.codecs.lucene50.Lucene50StoredFieldsFormat.fieldsWriter(Lucene50StoredFieldsFormat.java:183)
   [junit4]    > 	at org.apache.lucene.codecs.asserting.AssertingStoredFieldsFormat.fieldsWriter(AssertingStoredFieldsFormat.java:49)
   [junit4]    > 	at org.apache.lucene.index.DefaultIndexingChain.initStoredFieldsWriter(DefaultIndexingChain.java:81)
   [junit4]    > 	at org.apache.lucene.index.DefaultIndexingChain.startStoredFields(DefaultIndexingChain.java:258)
   [junit4]    > 	at org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:295)
   [junit4]    > 	at org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:234)
   [junit4]    > 	at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:450)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1477)
   [junit4]    > 	... 2 moreThrowable #9: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=1714, name=Thread-1637, state=RUNNABLE, group=TGRP-TestIndexWriterThreadsToSegments]
   [junit4]    > Caused by: java.lang.RuntimeException: java.nio.file.FileSystemException: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/test/J1/temp/lucene.index.TestIndexWriterThreadsToSegments_F72A04E52535E596-001/tempDir-001/_1ku.fdx: Too many open files
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([F72A04E52535E596]:0)
   [junit4]    > 	at org.apache.lucene.index.TestIndexWriterThreadsToSegments$2.run(TestIndexWriterThreadsToSegments.java:218)
   [junit4]    > Caused by: java.nio.file.FileSystemException: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/test/J1/temp/lucene.index.TestIndexWriterThreadsToSegments_F72A04E52535E596-001/tempDir-001/_1ku.fdx: Too many open files
   [junit4]    > 	at org.apache.lucene.mockfile.HandleLimitFS.onOpen(HandleLimitFS.java:49)
   [junit4]    > 	at org.apache.lucene.mockfile.HandleTrackingFS.callOpenHook(HandleTrackingFS.java:82)
   [junit4]    > 	at org.apache.lucene.mockfile.HandleTrackingFS.newOutputStream(HandleTrackingFS.java:161)
   [junit4]    > 	at org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:198)
   [junit4]    > 	at java.nio.file.Files.newOutputStream(Files.java:172)
   [junit4]    > 	at org.apache.lucene.store.FSDirectory$FSIndexOutput.<init>(FSDirectory.java:276)
   [junit4]    > 	at org.apache.lucene.store.FSDirectory.createOutput(FSDirectory.java:225)
   [junit4]    > 	at org.apache.lucene.store.MockDirectoryWrapper.createOutput(MockDirectoryWrapper.java:593)
   [junit4]    > 	at org.apache.lucene.store.LockValidatingDirectoryWrapper.createOutput(LockValidatingDirectoryWrapper.java:44)
   [junit4]    > 	at org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:43)
   [junit4]    > 	at org.apache.lucene.codecs.compressing.CompressingStoredFieldsWriter.<init>(CompressingStoredFieldsWriter.java:108)
   [junit4]    > 	at org.apache.lucene.codecs.compressing.CompressingStoredFieldsFormat.fieldsWriter(CompressingStoredFieldsFormat.java:128)
   [junit4]    > 	at org.apache.lucene.codecs.lucene50.Lucene50StoredFieldsFormat.fieldsWriter(Lucene50StoredFieldsFormat.java:183)
   [junit4]    > 	at org.apache.lucene.codecs.asserting.AssertingStoredFieldsFormat.fieldsWriter(AssertingStoredFieldsFormat.java:49)
   [junit4]    > 	at org.apache.lucene.index.DefaultIndexingChain.initStoredFieldsWriter(DefaultIndexingChain.java:81)
   [junit4]    > 	at org.apache.lucene.index.DefaultIndexingChain.startStoredFields(DefaultIndexingChain.java:258)
   [junit4]    > 	at org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:295)
   [junit4]    > 	at org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:234)
   [junit4]    > 	at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:450)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1477)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1256)
   [junit4]    > 	at org.apache.lucene.index.TestIndexWriterThreadsToSegments$2.run(TestIndexWriterThreadsToSegments.java:207)
   [junit4] Completed [366/414 (1!)] on J1 in 7220.20s, 3 tests, 10 errors <<< FAILURES!

[...truncated 170 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:801: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:738: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build.xml:50: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1456: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1010: There were test failures: 414 suites (1 ignored), 3345 tests, 9 suite-level errors, 1 error, 29 ignored (25 assumptions) [seed: F72A04E52535E596]

Total time: 157 minutes 50 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1044 - Still Failing

[Apache Jenkins Server <je...@builds.apache.org>]

Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1044/

2 tests failed.
FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Error from server at http://127.0.0.1:41741/_v/vj: Could not load collection from ZK:halfcollectionblocker

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:41741/_v/vj: Could not load collection from ZK:halfcollectionblocker
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.makeRequest(CollectionsAPIDistributedZkTest.java:295)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testErrorHandling(CollectionsAPIDistributedZkTest.java:412)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test(CollectionsAPIDistributedZkTest.java:162)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:965)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:940)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.cloud.RollingRestartTest.test

Error Message:
Unable to restart (#4): CloudJettyRunner [url=http://127.0.0.1:56785/be/c/collection1]

Stack Trace:
java.lang.AssertionError: Unable to restart (#4): CloudJettyRunner [url=http://127.0.0.1:56785/be/c/collection1]
	at __randomizedtesting.SeedInfo.seed([91E270C9DE67D054:19B64F13709BBDAC]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.apache.solr.cloud.RollingRestartTest.restartWithRolesTest(RollingRestartTest.java:104)
	at org.apache.solr.cloud.RollingRestartTest.test(RollingRestartTest.java:53)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:965)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:940)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 10372 lines...]
   [junit4] Suite: org.apache.solr.cloud.RollingRestartTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.RollingRestartTest_91E270C9DE67D054-001/init-core-data-001
   [junit4]   2> 169612 INFO  (SUITE-RollingRestartTest-seed#[91E270C9DE67D054]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false)
   [junit4]   2> 169613 INFO  (SUITE-RollingRestartTest-seed#[91E270C9DE67D054]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /be/c
   [junit4]   2> 169627 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 169628 INFO  (Thread-3945) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 169628 INFO  (Thread-3945) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 169728 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.ZkTestServer start zk server on port:55353
   [junit4]   2> 169729 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 169758 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 169762 INFO  (zkCallback-83-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@16b7755f name:ZooKeeperConnection Watcher:127.0.0.1:55353 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 169762 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 169762 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 169762 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 169766 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 169767 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 169773 INFO  (zkCallback-84-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2917ca48 name:ZooKeeperConnection Watcher:127.0.0.1:55353/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 169773 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 169774 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 169774 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 169777 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 169779 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 169781 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 169783 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 169783 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 169788 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 169788 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 169791 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 169791 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 169794 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 169794 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 169799 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 169799 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 169802 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 169803 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 169805 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 169805 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 169807 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 169807 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 169810 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 169810 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 169812 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 169812 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 169814 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 169814 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 170178 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.RollingRestartTest_91E270C9DE67D054-001/control-001/cores/collection1
   [junit4]   2> 170180 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 170183 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5ce1ccdd{/be/c,null,AVAILABLE}
   [junit4]   2> 170184 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.e.j.s.ServerConnector Started ServerConnector@6c50f0ce{HTTP/1.1}{127.0.0.1:39963}
   [junit4]   2> 170184 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.e.j.s.Server Started @173124ms
   [junit4]   2> 170184 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/be/c, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.RollingRestartTest_91E270C9DE67D054-001/control-001/cores, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.RollingRestartTest_91E270C9DE67D054-001/tempDir-001/control/data, hostPort=39963}
   [junit4]   2> 170185 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@4dfd245f
   [junit4]   2> 170185 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.RollingRestartTest_91E270C9DE67D054-001/control-001'
   [junit4]   2> 170185 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 170185 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 170186 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 170186 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 170188 INFO  (zkCallback-85-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@66490ef3 name:ZooKeeperConnection Watcher:127.0.0.1:55353/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 170188 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 170188 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 170190 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 170190 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.RollingRestartTest_91E270C9DE67D054-001/control-001/solr.xml
   [junit4]   2> 170204 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.RollingRestartTest_91E270C9DE67D054-001/control-001/cores
   [junit4]   2> 170205 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.CoreContainer New CoreContainer 1784043863
   [junit4]   2> 170205 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.RollingRestartTest_91E270C9DE67D054-001/control-001]
   [junit4]   2> 170205 WARN  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.RollingRestartTest_91E270C9DE67D054-001/control-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.RollingRestartTest_91E270C9DE67D054-001/control-001/lib
   [junit4]   2> 170205 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 170206 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 170207 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 170207 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 170207 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:55353/solr
   [junit4]   2> 170207 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 170207 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 170208 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 170210 INFO  (zkCallback-87-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1323271c name:ZooKeeperConnection Watcher:127.0.0.1:55353 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 170210 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 170211 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 170214 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [n:127.0.0.1:39963_be%2Fc    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 170216 INFO  (zkCallback-88-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6e13f60e name:ZooKeeperConnection Watcher:127.0.0.1:55353/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 170216 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [n:127.0.0.1:39963_be%2Fc    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 170218 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [n:127.0.0.1:39963_be%2Fc    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 170220 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [n:127.0.0.1:39963_be%2Fc    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 170224 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [n:127.0.0.1:39963_be%2Fc    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 170227 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [n:127.0.0.1:39963_be%2Fc    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 170229 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [n:127.0.0.1:39963_be%2Fc    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 170232 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [n:127.0.0.1:39963_be%2Fc    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 170235 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [n:127.0.0.1:39963_be%2Fc    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 170237 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [n:127.0.0.1:39963_be%2Fc    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 170239 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [n:127.0.0.1:39963_be%2Fc    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 170241 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [n:127.0.0.1:39963_be%2Fc    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:39963_be%2Fc
   [junit4]   2> 170242 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [n:127.0.0.1:39963_be%2Fc    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:39963_be%2Fc
   [junit4]   2> 170244 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [n:127.0.0.1:39963_be%2Fc    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 170246 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [n:127.0.0.1:39963_be%2Fc    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 170248 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [n:127.0.0.1:39963_be%2Fc    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 170249 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [n:127.0.0.1:39963_be%2Fc    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/95020415899533316-127.0.0.1:39963_be%2Fc-n_0000000000
   [junit4]   2> 170250 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [n:127.0.0.1:39963_be%2Fc    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:39963_be%2Fc
   [junit4]   2> 170250 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [n:127.0.0.1:39963_be%2Fc    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 170252 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [n:127.0.0.1:39963_be%2Fc    ] o.a.s.c.Overseer Overseer (id=95020415899533316-127.0.0.1:39963_be%2Fc-n_0000000000) starting
   [junit4]   2> 170254 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [n:127.0.0.1:39963_be%2Fc    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 170275 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [n:127.0.0.1:39963_be%2Fc    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 170275 INFO  (OverseerCollectionConfigSetProcessor-95020415899533316-127.0.0.1:39963_be%2Fc-n_0000000000) [n:127.0.0.1:39963_be%2Fc    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 170276 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [n:127.0.0.1:39963_be%2Fc    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 170278 INFO  (OverseerStateUpdate-95020415899533316-127.0.0.1:39963_be%2Fc-n_0000000000) [n:127.0.0.1:39963_be%2Fc    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 170314 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [n:127.0.0.1:39963_be%2Fc    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 170315 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [n:127.0.0.1:39963_be%2Fc    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 170316 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [n:127.0.0.1:39963_be%2Fc    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.RollingRestartTest_91E270C9DE67D054-001/control-001/cores
   [junit4]   2> 170317 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [n:127.0.0.1:39963_be%2Fc    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {coreNodeName=, loadOnStartup=true, transient=false, configSetProperties=configsetprops.json, schema=schema.xml, dataDir=data/, collection=control_collection, name=collection1, shard=, config=solrconfig.xml}
   [junit4]   2> 170317 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [n:127.0.0.1:39963_be%2Fc    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.RollingRestartTest_91E270C9DE67D054-001/control-001/cores/collection1
   [junit4]   2> 170318 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [n:127.0.0.1:39963_be%2Fc    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 170319 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [n:127.0.0.1:39963_be%2Fc    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 170319 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [n:127.0.0.1:39963_be%2Fc    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 170320 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 170320 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 170324 INFO  (OverseerStateUpdate-95020415899533316-127.0.0.1:39963_be%2Fc-n_0000000000) [n:127.0.0.1:39963_be%2Fc    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "node_name":"127.0.0.1:39963_be%2Fc",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "shard":null,
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "base_url":"http://127.0.0.1:39963/be/c"} current state version: 0
   [junit4]   2> 170324 INFO  (OverseerStateUpdate-95020415899533316-127.0.0.1:39963_be%2Fc-n_0000000000) [n:127.0.0.1:39963_be%2Fc    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "node_name":"127.0.0.1:39963_be%2Fc",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "shard":null,
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "base_url":"http://127.0.0.1:39963/be/c"}
   [junit4]   2> 170325 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 170325 INFO  (OverseerStateUpdate-95020415899533316-127.0.0.1:39963_be%2Fc-n_0000000000) [n:127.0.0.1:39963_be%2Fc    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 170325 INFO  (OverseerStateUpdate-95020415899533316-127.0.0.1:39963_be%2Fc-n_0000000000) [n:127.0.0.1:39963_be%2Fc    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 170328 INFO  (zkCallback-88-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 171325 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 171326 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 171327 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 171327 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 171328 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 171328 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.RollingRestartTest_91E270C9DE67D054-001/control-001/cores/collection1'
   [junit4]   2> 171328 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 171328 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 171331 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 171339 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 171354 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.5.0
   [junit4]   2> 171384 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection   x:collection1] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 171394 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 171670 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 171679 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 171682 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 171705 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 171710 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 171713 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 171714 WARN  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 171714 WARN  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 171714 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 171715 WARN  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 171715 WARN  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 171716 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 171717 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 171717 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore solr.StandardDirectoryFactory
   [junit4]   2> 171724 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.RollingRestartTest_91E270C9DE67D054-001/control-001/cores/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.RollingRestartTest_91E270C9DE67D054-001/control-001/cores/collection1/data/]
   [junit4]   2> 171724 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@531fb850
   [junit4]   2> 171725 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.RollingRestartTest_91E270C9DE67D054-001/control-001/cores/collection1/data
   [junit4]   2> 171726 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.RollingRestartTest_91E270C9DE67D054-001/control-001/cores/collection1/data/index/
   [junit4]   2> 171726 WARN  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.RollingRestartTest_91E270C9DE67D054-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 171726 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.RollingRestartTest_91E270C9DE67D054-001/control-001/cores/collection1/data/index
   [junit4]   2> 171727 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=50, maxMergeAtOnceExplicit=32, maxMergedSegmentMB=93.4091796875, floorSegmentMB=1.140625, forceMergeDeletesPctAllowed=19.040737700900713, segmentsPerTier=29.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.798895882218802
   [junit4]   2> 171735 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.RollingRestartTest_91E270C9DE67D054-001/control-001/cores/collection1/data/index,segFN=segments_1,generation=1}
   [junit4]   2> 171735 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 171736 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.RollingRestartTest_91E270C9DE67D054-001/control-001/cores/collection1/data/
   [junit4]   2> 171742 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 171742 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 171743 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 171743 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 171743 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 171743 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 171743 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 171743 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 171744 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 171744 WARN  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
   [junit4]   2> 171748 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 171749 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 171750 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 171752 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 171757 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/plugins,/admin/luke,/admin/mbeans,/dump,/admin/segments,/update/csv,standard,/schema,/replication,/config,/admin/properties,/update,/admin/system,/get,/update/json/docs,/admin/logging,/admin/ping,/update/json,/admin/threads,/admin/file
   [junit4]   2> 171759 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 171760 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 171760 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 171760 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 171760 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 171761 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=14, maxMergeAtOnceExplicit=27, maxMergedSegmentMB=73.013671875, floorSegmentMB=1.62890625, forceMergeDeletesPctAllowed=22.559599334711724, segmentsPerTier=47.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
   [junit4]   2> 171763 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.RollingRestartTest_91E270C9DE67D054-001/control-001/cores/collection1/data/index,segFN=segments_1,generation=1}
   [junit4]   2> 171763 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 171763 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@1a72b5ed[collection1] main
   [junit4]   2> 171763 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 171764 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 171764 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 171765 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 171766 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 171766 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 171766 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 171766 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 171766 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 171766 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 171767 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 171767 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 171767 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 171767 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1520326656622854144
   [junit4]   2> 171769 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 2.0ms to seed version buckets with highest version 1520326656622854144
   [junit4]   2> 171769 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 171770 INFO  (searcherExecutor-285-thread-1-processing-n:127.0.0.1:39963_be%2Fc s:shard1 c:control_collection x:collection1 r:core_node1) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@1a72b5ed[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 171770 INFO  (coreLoadExecutor-284-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 171771 INFO  (coreZkRegister-278-thread-1-processing-n:127.0.0.1:39963_be%2Fc s:shard1 c:control_collection x:collection1 r:core_node1) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:39963/be/c collection:control_collection shard:shard1
   [junit4]   2> 171773 INFO  (coreZkRegister-278-thread-1-processing-n:127.0.0.1:39963_be%2Fc s:shard1 c:control_collection x:collection1 r:core_node1) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 171778 INFO  (coreZkRegister-278-thread-1-processing-n:127.0.0.1:39963_be%2Fc s:shard1 c:control_collection x:collection1 r:core_node1) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/95020415899533316-core_node1-n_0000000000
   [junit4]   2> 171779 INFO  (coreZkRegister-278-thread-1-processing-n:127.0.0.1:39963_be%2Fc s:shard1 c:control_collection x:collection1 r:core_node1) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard=shard1 and weAreReplacement=false and leaderVoteWait=10000
   [junit4]   2> 171781 INFO  (coreZkRegister-278-thread-1-processing-n:127.0.0.1:39963_be%2Fc s:shard1 c:control_collection x:collection1 r:core_node1) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 171782 INFO  (coreZkRegister-278-thread-1-processing-n:127.0.0.1:39963_be%2Fc s:shard1 c:control_collection x:collection1 r:core_node1) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 171782 INFO  (coreZkRegister-278-thread-1-processing-n:127.0.0.1:39963_be%2Fc s:shard1 c:control_collection x:collection1 r:core_node1) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:39963/be/c/collection1/
   [junit4]   2> 171782 INFO  (coreZkRegister-278-thread-1-processing-n:127.0.0.1:39963_be%2Fc s:shard1 c:control_collection x:collection1 r:core_node1) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 171782 INFO  (coreZkRegister-278-thread-1-processing-n:127.0.0.1:39963_be%2Fc s:shard1 c:control_collection x:collection1 r:core_node1) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:39963/be/c/collection1/ has no replicas
   [junit4]   2> 171782 INFO  (OverseerStateUpdate-95020415899533316-127.0.0.1:39963_be%2Fc-n_0000000000) [n:127.0.0.1:39963_be%2Fc    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 171783 INFO  (coreZkRegister-278-thread-1-processing-n:127.0.0.1:39963_be%2Fc s:shard1 c:control_collection x:collection1 r:core_node1) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 171788 INFO  (coreZkRegister-278-thread-1-processing-n:127.0.0.1:39963_be%2Fc s:shard1 c:control_collection x:collection1 r:core_node1) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 171791 INFO  (coreZkRegister-278-thread-1-processing-n:127.0.0.1:39963_be%2Fc s:shard1 c:control_collection x:collection1 r:core_node1) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:39963/be/c/collection1/ shard1
   [junit4]   2> 171792 INFO  (OverseerStateUpdate-95020415899533316-127.0.0.1:39963_be%2Fc-n_0000000000) [n:127.0.0.1:39963_be%2Fc    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:39963/be/c",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 1
   [junit4]   2> 171821 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 171822 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 171824 INFO  (zkCallback-90-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@35fda302 name:ZooKeeperConnection Watcher:127.0.0.1:55353/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 171824 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 171824 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 171824 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 171829 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 171896 INFO  (zkCallback-88-thread-1-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 171896 INFO  (zkCallback-90-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 171943 INFO  (coreZkRegister-278-thread-1-processing-n:127.0.0.1:39963_be%2Fc s:shard1 c:control_collection x:collection1 r:core_node1) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:39963/be/c/collection1/ and leader is http://127.0.0.1:39963/be/c/collection1/
   [junit4]   2> 171943 INFO  (coreZkRegister-278-thread-1-processing-n:127.0.0.1:39963_be%2Fc s:shard1 c:control_collection x:collection1 r:core_node1) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:39963/be/c
   [junit4]   2> 171943 INFO  (coreZkRegister-278-thread-1-processing-n:127.0.0.1:39963_be%2Fc s:shard1 c:control_collection x:collection1 r:core_node1) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 171943 INFO  (coreZkRegister-278-thread-1-processing-n:127.0.0.1:39963_be%2Fc s:shard1 c:control_collection x:collection1 r:core_node1) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 171943 INFO  (coreZkRegister-278-thread-1-processing-n:127.0.0.1:39963_be%2Fc s:shard1 c:control_collection x:collection1 r:core_node1) [n:127.0.0.1:39963_be%2Fc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 171946 INFO  (OverseerStateUpdate-95020415899533316-127.0.0.1:39963_be%2Fc-n_0000000000) [n:127.0.0.1:39963_be%2Fc    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "node_name":"127.0.0.1:39963_be%2Fc",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "base_url":"http://127.0.0.1:39963/be/c"} current state version: 2
   [junit4]   2> 171947 INFO  (OverseerStateUpdate-95020415899533316-127.0.0.1:39963_be%2Fc-n_0000000000) [n:127.0.0.1:39963_be%2Fc    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "node_name":"127.0.0.1:39963_be%2Fc",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "base_url":"http://127.0.0.1:39963/be/c"}
   [junit4]   2> 172051 INFO  (zkCallback-90-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 172051 INFO  (zkCallback-88-thread-2-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 172190 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.RollingRestartTest_91E270C9DE67D054-001/shard-1-001/cores/collection1
   [junit4]   2> 172191 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.RollingRestartTest_91E270C9DE67D054-001/shard-1-001
   [junit4]   2> 172191 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 172194 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@3951e1e6{/be/c,null,AVAILABLE}
   [junit4]   2> 172194 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.e.j.s.ServerConnector Started ServerConnector@4111f10d{HTTP/1.1}{127.0.0.1:57469}
   [junit4]   2> 172195 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.e.j.s.Server Started @175134ms
   [junit4]   2> 172195 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostPort=57469, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.RollingRestartTest_91E270C9DE67D054-001/shard-1-001/cores, hostContext=/be/c, solrconfig=solrconfig.xml, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.RollingRestartTest_91E270C9DE67D054-001/tempDir-001/jetty1}
   [junit4]   2> 172195 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@4dfd245f
   [junit4]   2> 172195 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.RollingRestartTest_91E270C9DE67D054-001/shard-1-001'
   [junit4]   2> 172195 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 172195 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 172196 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 172196 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 172198 INFO  (zkCallback-91-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@32c8d2ed name:ZooKeeperConnection Watcher:127.0.0.1:55353/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 172198 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 172199 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 172200 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 172200 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.RollingRestartTest_91E270C9DE67D054-001/shard-1-001/solr.xml
   [junit4]   2> 172217 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.RollingRestartTest_91E270C9DE67D054-001/shard-1-001/cores
   [junit4]   2> 172218 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.CoreContainer New CoreContainer 1400400768
   [junit4]   2> 172218 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.RollingRestartTest_91E270C9DE67D054-001/shard-1-001]
   [junit4]   2> 172218 WARN  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.RollingRestartTest_91E270C9DE67D054-001/shard-1-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.RollingRestartTest_91E270C9DE67D054-001/shard-1-001/lib
   [junit4]   2> 172219 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 172220 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 172221 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 172221 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 172221 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:55353/solr
   [junit4]   2> 172221 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 172221 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 172222 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 172223 INFO  (zkCallback-93-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@73c68f60 name:ZooKeeperConnection Watcher:127.0.0.1:55353 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 172224 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 172224 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 172227 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [n:127.0.0.1:57469_be%2Fc    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 172230 INFO  (zkCallback-94-thread-1-processing-n:127.0.0.1:57469_be%2Fc) [n:127.0.0.1:57469_be%2Fc    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7212f0ad name:ZooKeeperConnection Watcher:127.0.0.1:55353/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 172230 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [n:127.0.0.1:57469_be%2Fc    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 172238 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [n:127.0.0.1:57469_be%2Fc    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 173247 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [n:127.0.0.1:57469_be%2Fc    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:57469_be%2Fc
   [junit4]   2> 173247 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [n:127.0.0.1:57469_be%2Fc    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:57469_be%2Fc
   [junit4]   2> 173249 INFO  (zkCallback-94-thread-1-processing-n:127.0.0.1:57469_be%2Fc) [n:127.0.0.1:57469_be%2Fc    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 173250 INFO  (zkCallback-90-thread-1) [    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 173250 INFO  (zkCallback-88-thread-2-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 173251 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [n:127.0.0.1:57469_be%2Fc    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 173252 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [n:127.0.0.1:57469_be%2Fc    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/95020415899533320-127.0.0.1:57469_be%2Fc-n_0000000001
   [junit4]   2> 173254 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [n:127.0.0.1:57469_be%2Fc    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/95020415899533316-127.0.0.1:39963_be%2Fc-n_0000000000 to know if I could be the leader
   [junit4]   2> 173275 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [n:127.0.0.1:57469_be%2Fc    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 173275 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [n:127.0.0.1:57469_be%2Fc    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 173276 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [n:127.0.0.1:57469_be%2Fc    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.RollingRestartTest_91E270C9DE67D054-001/shard-1-001/cores
   [junit4]   2> 173277 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [n:127.0.0.1:57469_be%2Fc    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {coreNodeName=, loadOnStartup=true, config=solrconfig.xml, transient=false, name=collection1, shard=, collection=collection1, dataDir=data/, schema=schema.xml, configSetProperties=configsetprops.json}
   [junit4]   2> 173277 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [n:127.0.0.1:57469_be%2Fc    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.RollingRestartTest_91E270C9DE67D054-001/shard-1-001/cores/collection1
   [junit4]   2> 173278 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [n:127.0.0.1:57469_be%2Fc    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 173280 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [n:127.0.0.1:57469_be%2Fc    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 173280 INFO  (coreLoadExecutor-295-thread-1-processing-n:127.0.0.1:57469_be%2Fc) [n:127.0.0.1:57469_be%2Fc c:collection1   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 173280 INFO  (TEST-RollingRestartTest.test-seed#[91E270C9DE67D054]) [n:127.0.0.1:57469_be%2Fc    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 173280 INFO  (coreLoadExecutor-295-thread-1-processing-n:127.0.0.1:57469_be%2Fc) [n:127.0.0.1:57469_be%2Fc c:collection1   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 173283 INFO  (coreLoadExecutor-295-thread-1-processing-n:127.0.0.1:57469_be%2Fc) [n:127.0.0.1:57469_be%2Fc c:collection1   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 173284 INFO  (OverseerStateUpdate-95020415899533316-127.0.0.1:39963_be%2Fc-n_0000000000) [n:127.0.0.1:39963_be%2Fc    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "node_name":"127.0.0.1:57469_be%2Fc",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "shard":null,
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "base_url":"http://127.0.0.1:57469/be/c"} current state version: 3
   [junit4]   2> 173284 INFO  (OverseerStateUpdate-95020415899533316-127.0.0.1:39963_be%2Fc-n_0000000000) [n:127.0.0.1:39963_be%2Fc    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "node_name":"127.0.0.1:57469_be%2Fc",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "shard":null,
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "base_url":"http://127.0.0.1:57469/be/c"}
   [junit4]   2> 173284 INFO  (OverseerStateUpdate-95020415899533316-127.0.0.1:39963_be%2Fc-n_0000000000) [n:127.0.0.1:39963_be%2Fc    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 173284 INFO  (OverseerStateUpdate-95020415899533316-127.0.0.1:39963_be%2Fc-n_0000000000) [n:127.0.0.1:39963_be%2Fc    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard2
   [junit4]   2> 173389 INFO  (zkCallback-88-thread-2-processing-n:127.0.0.1:39963_be%2Fc) [n:127.0.0.1:39963_be%2Fc    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 173389 INFO  (zkCallback-94-thread-1-processing-n:127.0.0.1:57469_be%2Fc) [n:127.0.0.1:57469_be%2Fc    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 173389 INFO  (zkCallback-90-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 174284 INFO  (coreLoadExecutor-295-thread-1-processing-n:127.0.0.1:57469_be%2Fc) [n:127.0.0.1:57469_be%2Fc c:collection1   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 174285 INFO  (coreLoadExecutor-295-thread-1-processing-n:127.0.0.1:57469_be%2Fc) [n:127.0.0.1:57469_be%2Fc c:collection1   x:collection1] o.a.s.c.ZkController Check for collection zkNode:collection1
   [junit4]   2> 174285 INFO  (coreLoadExecutor-295-thread-1-processing-n:127.0.0.1:57469_be%2Fc) [n:127.0.0.1:57469_be%2Fc c:collection1   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 174285 INFO  (coreLoadExecutor-295-thread-1-processing-n:127.0.0.1:57469_be%2Fc) [n:127.0.0.1:57469_be%2Fc c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/collection1
   [junit4]   2> 174286 INFO  (coreLoadExecutor-295-thread-1-processing-n:127.0.0.1:57469_be%2Fc) [n:127.0.0.1:57469_be%2Fc c:collection1   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 174286 INFO  (coreLoadExecutor-295-thread-1-processing-n:127.0.0.1:57469_be%2Fc) [n:127.0.0.1:57469_be%2Fc c:collection1   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.RollingRestartTest_91E270C9DE67D054-001/shard-1-001/cores/collection1'
   [junit4]   2> 174287 INFO  (coreLoadExecutor-295-thread-1-processing-n:127.0.0.1:57469_be%2Fc) [n:127.0.0.1:57469_be%2Fc c:collection1   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 174287 INFO  (coreLoadExecutor-295-thread-1-processing-n:127.0.0.1:57469_be%2Fc) [n:127.0.0.1:57469_be%2Fc c:collection1   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 174290 INFO  (coreLoadExecutor-295-thread-1-processing-n:127.0.0.1:57469_be%2Fc) [n:127.0.0.1:57469_be%2Fc c:collection1   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 174298 INFO  (coreLoadExecutor-295-thread-1-processing-n:127.0.0.1:57469_be%2Fc) [n:127.0.0.1:57469_be%2Fc c:collection1   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 174315 INFO  (coreLoadExecutor-295-thread-1-processing-n:127.0.0.1:57469_be%2Fc) [n:127.0.0.1:57469_be%2Fc c:collection1   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.5.0
   [junit4]   2> 174362 INFO  (coreLoadExecutor-295-thread-1-processing-n:127.0.0.1:57469_be%2Fc) [n:127.0.0.1:57469_be%2Fc c:collection1   x:collection1] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 174371 INFO  (coreLoadExecutor-295-thread-1-processing-n:127.0.0.1:57469_be%2Fc) [n:127.0.0.1:57469_be%2Fc c:collection1   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 174633 INFO  (coreLoadExecutor-295-thread-1-processing-n:127.0.0.1:57469_be%2Fc) [n:127.0.0.1:57469_be%2Fc c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 174643 INFO  (coreLoadExecutor-295-thread-1-processing-n:127.0.0.1:57469_be%2Fc) [n:127.0.0.1:57469_be%2Fc c:collection1   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 174646 INFO  (coreLoadExecutor-295-thread-1-processing-n:127.0.0.1:57469_be%2Fc) [n:127.0.0.1:57469_be%2Fc c:collection1   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 174663 INFO  (coreLoadExecutor-295-thread-1-processing-n:127.0.0.1:57469_be%2Fc) [n:127.0.0.1:57469_be%2Fc c:collection1   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 174668 INFO  (coreLoadExecutor-295-thread-1-processing-n:127.0.0.1:57469_be%2Fc) [n:127.0.0.1:57469_be%2Fc c:collection1   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 174671 INFO  (coreLoadExecutor-295-thread-1-processing-n:127.0.0.1:57469_be%2Fc) [n:127.0.0.1:57469_be%2Fc c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 174672 WARN  (coreLoadExecutor-295-thread-1-processing-n:127.0.0.1:57469_be%2Fc) [n:127.0.0.1:57469_be%2Fc c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 174672 WARN  (coreLoadExecutor-295-thread-1-processing-n:127.0.0.1:57469_be%2Fc) [n:127.0.0.1:57469_be%2Fc c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 174673 INFO  (coreLoadExecutor-295-thread-1-processing-n:127.0.0.1:5

[...truncated too long message...]

p://127.0.0.1:48329/_v/vj, http://127.0.0.1:44086/_v/vj, http://127.0.0.1:35231/_v/vj, http://127.0.0.1:46864/_v/vj]
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([91E270C9DE67D054]:0)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:891)
   [junit4]    > Caused by: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:41741/_v/vj, http://127.0.0.1:48329/_v/vj, http://127.0.0.1:44086/_v/vj, http://127.0.0.1:35231/_v/vj, http://127.0.0.1:46864/_v/vj]
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:352)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1100)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:871)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:807)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:888)
   [junit4]    > Caused by: org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:46864/_v/vj
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:585)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
   [junit4]    > 	... 5 more
   [junit4]    > Caused by: java.net.SocketTimeoutException: Read timed out
   [junit4]    > 	at java.net.SocketInputStream.socketRead0(Native Method)
   [junit4]    > 	at java.net.SocketInputStream.read(SocketInputStream.java:152)
   [junit4]    > 	at java.net.SocketInputStream.read(SocketInputStream.java:122)
   [junit4]    > 	at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:160)
   [junit4]    > 	at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:84)
   [junit4]    > 	at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:273)
   [junit4]    > 	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
   [junit4]    > 	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
   [junit4]    > 	at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
   [junit4]    > 	at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283)
   [junit4]    > 	at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251)
   [junit4]    > 	at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197)
   [junit4]    > 	at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)
   [junit4]    > 	at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)
   [junit4]    > 	at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685)
   [junit4]    > 	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487)
   [junit4]    > 	at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
   [junit4]    > 	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
   [junit4]    > 	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
   [junit4]    > 	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:480)
   [junit4]    > 	... 9 more
   [junit4]   2> 1977347 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[91E270C9DE67D054]-worker) [n:127.0.0.1:44086__v%2Fvj c:awholynewstresscollection_collection5_1 s:shard5 r:core_node12 x:awholynewstresscollection_collection5_1_shard5_replica3] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_91E270C9DE67D054-001
   [junit4]   2> Dec 12, 2015 5:31:30 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=FastDecompressionCompressingStoredFields(storedFieldsFormat=CompressingStoredFieldsFormat(compressionMode=FAST_DECOMPRESSION, chunkSize=4892, maxDocsPerChunk=810, blockSize=4), termVectorsFormat=CompressingTermVectorsFormat(compressionMode=FAST_DECOMPRESSION, chunkSize=4892, blockSize=4)), sim=RandomSimilarityProvider(queryNorm=false,coord=yes): {}, locale=ko_KR, timezone=Africa/Porto-Novo
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_80 (64-bit)/cpus=4,threads=1,free=92832704,total=477102080
   [junit4]   2> NOTE: All tests run in this JVM: [MBeansHandlerTest, ZkCLITest, ChaosMonkeySafeLeaderTest, OutOfBoxZkACLAndCredentialsProvidersTest, UpdateRequestProcessorFactoryTest, QueryResultKeyTest, TestQueryUtils, SpellPossibilityIteratorTest, TestCollapseQParserPlugin, TestFastLRUCache, TestDFRSimilarityFactory, DocumentBuilderTest, TestSolrCloudWithKerberosAlt, DeleteLastCustomShardedReplicaTest, ChaosMonkeyNothingIsSafeTest, TestManagedResource, SortSpecParsingTest, DirectSolrSpellCheckerTest, TestLMJelinekMercerSimilarityFactory, DocExpirationUpdateProcessorFactoryTest, ClusterStateUpdateTest, TestTrie, TestFileDictionaryLookup, SpellCheckCollatorTest, TestQueryTypes, FastVectorHighlighterTest, HdfsDirectoryTest, TestFieldTypeResource, ConnectionManagerTest, SolrInfoMBeanTest, SchemaVersionSpecificBehaviorTest, HdfsThreadLeakTest, TestQuerySenderListener, DistribCursorPagingTest, TermsComponentTest, WordBreakSolrSpellCheckerTest, TimeZoneUtilsTest, ConvertedLegacyTest, StressHdfsTest, TestReplicationHandlerBackup, TestLFUCache, DistributedFacetPivotSmallAdvancedTest, TestDistribDocBasedVersion, NotRequiredUniqueKeyTest, TestMiniSolrCloudClusterBase, TestScoreJoinQPNoScore, TestTrieFacet, SolrIndexConfigTest, ExplicitHLLTest, SuggesterFSTTest, TestSchemaResource, RecoveryAfterSoftCommitTest, SolrPluginUtilsTest, TestReRankQParserPlugin, TestExactStatsCache, TestMaxScoreQueryParser, TestOrdValues, SolrXmlInZkTest, SharedFSAutoReplicaFailoverUtilsTest, BlockCacheTest, TestOverriddenPrefixQueryForCustomFieldType, TestSchemaManager, FileUtilsTest, TestCloudSchemaless, TestFieldSortValues, MultiTermTest, AssignTest, OpenExchangeRatesOrgProviderTest, ZkSolrClientTest, TestCodecSupport, TestFieldCollectionResource, BasicFunctionalityTest, TestChildDocTransformer, TestSimpleTrackingShardHandler, TestCursorMarkWithoutUniqueKey, DistributedTermsComponentTest, CloudMLTQParserTest, SoftAutoCommitTest, ShardRoutingTest, AddSchemaFieldsUpdateProcessorFactoryTest, DistanceFunctionTest, DocValuesMissingTest, TestBulkSchemaConcurrent, TestCloudPivotFacet, DistributedMLTComponentTest, ConjunctionSolrSpellCheckerTest, SimplePostToolTest, TestRandomForceMergePolicy, TermVectorComponentTest, TestBinaryResponseWriter, BasicDistributedZkTest, BasicDistributedZk2Test, CollectionsAPIDistributedZkTest]
   [junit4] Completed [239/555 (2!)] on J1 in 532.01s, 1 test, 1 error <<< FAILURES!

[...truncated 1008 lines...]
   [junit4] JVM J1: stdout was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J1-20151212_045830_148.sysout
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] java.lang.OutOfMemoryError: GC overhead limit exceeded
   [junit4] Dumping heap to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/heapdumps/java_pid7675.hprof ...
   [junit4] Heap dump file created [613583618 bytes in 8.932 secs]
   [junit4] <<< JVM J1: EOF ----

[...truncated 23 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:801: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:738: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1456: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1010: There were test failures: 555 suites (7 ignored), 2192 tests, 1 error, 1 failure, 33 ignored (18 assumptions) [seed: 91E270C9DE67D054]

Total time: 225 minutes 55 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1043 - Still Failing

[Apache Jenkins Server <je...@builds.apache.org>]

Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1043/

2 tests failed.
FAILED:  org.apache.solr.handler.TestReplicationHandler.doTestReplicateAfterCoreReload

Error Message:
expected:<[{indexVersion=1449825644507,generation=2,filelist=[_2njz.fdt, _2njz.fdx, _2njz.fnm, _2njz.nvd, _2njz.nvm, _2njz.si, _2njz_MockRandom_0.doc, _2njz_MockRandom_0.sd, _2njz_MockRandom_0.tbk, _2njz_MockRandom_0.tix, _2nve.cfe, _2nve.cfs, _2nve.si, _2nvh.fdt, _2nvh.fdx, _2nvh.fnm, _2nvh.nvd, _2nvh.nvm, _2nvh.si, _2nvh_MockRandom_0.doc, _2nvh_MockRandom_0.sd, _2nvh_MockRandom_0.tib, _2nvh_MockRandom_0.tii, _2nvn.cfe, _2nvn.cfs, _2nvn.si, _2nvp.fdt, _2nvp.fdx, _2nvp.fnm, _2nvp.nvd, _2nvp.nvm, _2nvp.si, _2nvp_MockRandom_0.doc, _2nvp_MockRandom_0.sd, _2nvp_MockRandom_0.tim, _2nvp_MockRandom_0.tip, _2nvq.cfe, _2nvq.cfs, _2nvq.si, _2nvr.fdt, _2nvr.fdx, _2nvr.fnm, _2nvr.nvd, _2nvr.nvm, _2nvr.si, _2nvr_MockRandom_0.doc, _2nvr_MockRandom_0.sd, _2nvr_MockRandom_0.tbk, _2nvr_MockRandom_0.tix, _2nwm.fdt, _2nwm.fdx, _2nwm.fnm, _2nwm.nvd, _2nwm.nvm, _2nwm.si, _2nwm_MockRandom_0.doc, _2nwm_MockRandom_0.sd, _2nwm_MockRandom_0.tmp, _2nwp.cfe, _2nwp.cfs, _2nwp.si, _2nwq.fdt, _2nwq.fdx, _2nwq.fnm, _2nwq.nvd, _2nwq.nvm, _2nwq.si, _2nwq_MockRandom_0.doc, _2nwq_MockRandom_0.sd, _2nwq_MockRandom_0.tbk, _2nwq_MockRandom_0.tix, _2nwr.fdt, _2nwr.fdx, _2nwr.fnm, _2nwr.nvd, _2nwr.nvm, _2nwr.si, _2nwr_MockRandom_0.doc, _2nwr_MockRandom_0.sd, _2nwr_MockRandom_0.tio, _2nwr_MockRandom_0.tipo, _2nws.fdt, _2nws.fdx, _2nws.fnm, _2nws.nvd, _2nws.nvm, _2nws.si, _2nws_MockRandom_0.doc, _2nws_MockRandom_0.sd, _2nws_MockRandom_0.tio, _2nws_MockRandom_0.tipo, _2nwu.fdt, _2nwu.fdx, _2nwu.fnm, _2nwu.nvd, _2nwu.nvm, _2nwu.si, _2nwu_MockRandom_0.doc, _2nwu_MockRandom_0.sd, _2nwu_MockRandom_0.tim, _2nwu_MockRandom_0.tip, _2nwx.fdt, _2nwx.fdx, _2nwx.fnm, _2nwx.nvd, _2nwx.nvm, _2nwx.si, _2nwx_MockRandom_0.doc, _2nwx_MockRandom_0.sd, _2nwx_MockRandom_0.tio, _2nwx_MockRandom_0.tipo, _2nx1.fdt, _2nx1.fdx, _2nx1.fnm, _2nx1.nvd, _2nx1.nvm, _2nx1.si, _2nx1_MockRandom_0.doc, _2nx1_MockRandom_0.sd, _2nx1_MockRandom_0.tmp, _2nx2.fdt, _2nx2.fdx, _2nx2.fnm, _2nx2.nvd, _2nx2.nvm, _2nx2.si, _2nx2_MockRandom_0.doc, _2nx2_MockRandom_0.sd, _2nx2_MockRandom_0.tib, _2nx2_MockRandom_0.tiv, _2nx3.cfe, _2nx3.cfs, _2nx3.si, _2nx4.fdt, _2nx4.fdx, _2nx4.fnm, _2nx4.nvd, _2nx4.nvm, _2nx4.si, _2nx4_MockRandom_0.doc, _2nx4_MockRandom_0.sd, _2nx4_MockRandom_0.tbk, _2nx4_MockRandom_0.tix, _2nx5.fdt, _2nx5.fdx, _2nx5.fnm, _2nx5.nvd, _2nx5.nvm, _2nx5.si, _2nx5_MockRandom_0.doc, _2nx5_MockRandom_0.sd, _2nx5_MockRandom_0.tmp, _2nx6.fdt, _2nx6.fdx, _2nx6.fnm, _2nx6.nvd, _2nx6.nvm, _2nx6.si, _2nx6_MockRandom_0.doc, _2nx6_MockRandom_0.sd, _2nx6_MockRandom_0.tib, _2nx6_MockRandom_0.tii, _2nx7.fdt, _2nx7.fdx, _2nx7.fnm, _2nx7.nvd, _2nx7.nvm, _2nx7.si, _2nx7_MockRandom_0.doc, _2nx7_MockRandom_0.sd, _2nx7_MockRandom_0.tbk, _2nx7_MockRandom_0.tix, _2nx8.fdt, _2nx8.fdx, _2nx8.fnm, _2nx8.nvd, _2nx8.nvm, _2nx8.si, _2nx8_MockRandom_0.doc, _2nx8_MockRandom_0.sd, _2nx8_MockRandom_0.tbk, _2nx8_MockRandom_0.tix, _2nx9.cfe, _2nx9.cfs, _2nx9.si, _2nxa.fdt, _2nxa.fdx, _2nxa.fnm, _2nxa.nvd, _2nxa.nvm, _2nxa.si, _2nxa_MockRandom_0.doc, _2nxa_MockRandom_0.sd, _2nxa_MockRandom_0.tio, _2nxa_MockRandom_0.tipo, _2nxb.fdt, _2nxb.fdx, _2nxb.fnm, _2nxb.nvd, _2nxb.nvm, _2nxb.si, _2nxb_MockRandom_0.doc, _2nxb_MockRandom_0.sd, _2nxb_MockRandom_0.tmp, _2nxc.fdt, _2nxc.fdx, _2nxc.fnm, _2nxc.nvd, _2nxc.nvm, _2nxc.si, _2nxc_MockRandom_0.doc, _2nxc_MockRandom_0.sd, _2nxc_MockRandom_0.tio, _2nxc_MockRandom_0.tipo, _2nxd.fdt, _2nxd.fdx, _2nxd.fnm, _2nxd.nvd, _2nxd.nvm, _2nxd.si, _2nxd_MockRandom_0.doc, _2nxd_MockRandom_0.sd, _2nxd_MockRandom_0.tim, _2nxd_MockRandom_0.tip, _2nxe.fdt, _2nxe.fdx, _2nxe.fnm, _2nxe.nvd, _2nxe.nvm, _2nxe.si, _2nxe_MockRandom_0.doc, _2nxe_MockRandom_0.sd, _2nxe_MockRandom_0.tbk, _2nxe_MockRandom_0.tix, _2nxf.fdt, _2nxf.fdx, _2nxf.fnm, _2nxf.nvd, _2nxf.nvm, _2nxf.si, _2nxf_MockRandom_0.doc, _2nxf_MockRandom_0.sd, _2nxf_MockRandom_0.tmp, _2nxh.fdt, _2nxh.fdx, _2nxh.fnm, _2nxh.nvd, _2nxh.nvm, _2nxh.si, _2nxh_MockRandom_0.doc, _2nxh_MockRandom_0.sd, _2nxh_MockRandom_0.tim, _2nxh_MockRandom_0.tip, _2nxi.fdt, _2nxi.fdx, _2nxi.fnm, _2nxi.nvd, _2nxi.nvm, _2nxi.si, _2nxi_MockRandom_0.doc, _2nxi_MockRandom_0.sd, _2nxi_MockRandom_0.tim, _2nxi_MockRandom_0.tip, _2nxj.fdt, _2nxj.fdx, _2nxj.fnm, _2nxj.nvd, _2nxj.nvm, _2nxj.si, _2nxj_MockRandom_0.doc, _2nxj_MockRandom_0.sd, _2nxj_MockRandom_0.tib, _2nxj_MockRandom_0.tii, _2nxk.fdt, _2nxk.fdx, _2nxk.fnm, _2nxk.nvd, _2nxk.nvm, _2nxk.si, _2nxk_MockRandom_0.doc, _2nxk_MockRandom_0.sd, _2nxk_MockRandom_0.tim, _2nxk_MockRandom_0.tip, segments_2]}]> but was:<[{indexVersion=1449825644507,generation=2,filelist=[_2njz.fdt, _2njz.fdx, _2njz.fnm, _2njz.nvd, _2njz.nvm, _2njz.si, _2njz_MockRandom_0.doc, _2njz_MockRandom_0.sd, _2njz_MockRandom_0.tbk, _2njz_MockRandom_0.tix, _2nve.cfe, _2nve.cfs, _2nve.si, _2nvh.fdt, _2nvh.fdx, _2nvh.fnm, _2nvh.nvd, _2nvh.nvm, _2nvh.si, _2nvh_MockRandom_0.doc, _2nvh_MockRandom_0.sd, _2nvh_MockRandom_0.tib, _2nvh_MockRandom_0.tii, _2nvn.cfe, _2nvn.cfs, _2nvn.si, _2nvp.fdt, _2nvp.fdx, _2nvp.fnm, _2nvp.nvd, _2nvp.nvm, _2nvp.si, _2nvp_MockRandom_0.doc, _2nvp_MockRandom_0.sd, _2nvp_MockRandom_0.tim, _2nvp_MockRandom_0.tip, _2nvq.cfe, _2nvq.cfs, _2nvq.si, _2nvr.fdt, _2nvr.fdx, _2nvr.fnm, _2nvr.nvd, _2nvr.nvm, _2nvr.si, _2nvr_MockRandom_0.doc, _2nvr_MockRandom_0.sd, _2nvr_MockRandom_0.tbk, _2nvr_MockRandom_0.tix, _2nwm.fdt, _2nwm.fdx, _2nwm.fnm, _2nwm.nvd, _2nwm.nvm, _2nwm.si, _2nwm_MockRandom_0.doc, _2nwm_MockRandom_0.sd, _2nwm_MockRandom_0.tmp, _2nwp.cfe, _2nwp.cfs, _2nwp.si, _2nwq.fdt, _2nwq.fdx, _2nwq.fnm, _2nwq.nvd, _2nwq.nvm, _2nwq.si, _2nwq_MockRandom_0.doc, _2nwq_MockRandom_0.sd, _2nwq_MockRandom_0.tbk, _2nwq_MockRandom_0.tix, _2nwr.fdt, _2nwr.fdx, _2nwr.fnm, _2nwr.nvd, _2nwr.nvm, _2nwr.si, _2nwr_MockRandom_0.doc, _2nwr_MockRandom_0.sd, _2nwr_MockRandom_0.tio, _2nwr_MockRandom_0.tipo, _2nws.fdt, _2nws.fdx, _2nws.fnm, _2nws.nvd, _2nws.nvm, _2nws.si, _2nws_MockRandom_0.doc, _2nws_MockRandom_0.sd, _2nws_MockRandom_0.tio, _2nws_MockRandom_0.tipo, _2nwu.fdt, _2nwu.fdx, _2nwu.fnm, _2nwu.nvd, _2nwu.nvm, _2nwu.si, _2nwu_MockRandom_0.doc, _2nwu_MockRandom_0.sd, _2nwu_MockRandom_0.tim, _2nwu_MockRandom_0.tip, _2nwx.fdt, _2nwx.fdx, _2nwx.fnm, _2nwx.nvd, _2nwx.nvm, _2nwx.si, _2nwx_MockRandom_0.doc, _2nwx_MockRandom_0.sd, _2nwx_MockRandom_0.tio, _2nwx_MockRandom_0.tipo, _2nx1.fdt, _2nx1.fdx, _2nx1.fnm, _2nx1.nvd, _2nx1.nvm, _2nx1.si, _2nx1_MockRandom_0.doc, _2nx1_MockRandom_0.sd, _2nx1_MockRandom_0.tmp, _2nx2.fdt, _2nx2.fdx, _2nx2.fnm, _2nx2.nvd, _2nx2.nvm, _2nx2.si, _2nx2_MockRandom_0.doc, _2nx2_MockRandom_0.sd, _2nx2_MockRandom_0.tib, _2nx2_MockRandom_0.tiv, _2nx3.cfe, _2nx3.cfs, _2nx3.si, _2nx4.fdt, _2nx4.fdx, _2nx4.fnm, _2nx4.nvd, _2nx4.nvm, _2nx4.si, _2nx4_MockRandom_0.doc, _2nx4_MockRandom_0.sd, _2nx4_MockRandom_0.tbk, _2nx4_MockRandom_0.tix, _2nx5.fdt, _2nx5.fdx, _2nx5.fnm, _2nx5.nvd, _2nx5.nvm, _2nx5.si, _2nx5_MockRandom_0.doc, _2nx5_MockRandom_0.sd, _2nx5_MockRandom_0.tmp, _2nx6.fdt, _2nx6.fdx, _2nx6.fnm, _2nx6.nvd, _2nx6.nvm, _2nx6.si, _2nx6_MockRandom_0.doc, _2nx6_MockRandom_0.sd, _2nx6_MockRandom_0.tib, _2nx6_MockRandom_0.tii, _2nx7.fdt, _2nx7.fdx, _2nx7.fnm, _2nx7.nvd, _2nx7.nvm, _2nx7.si, _2nx7_MockRandom_0.doc, _2nx7_MockRandom_0.sd, _2nx7_MockRandom_0.tbk, _2nx7_MockRandom_0.tix, _2nx8.fdt, _2nx8.fdx, _2nx8.fnm, _2nx8.nvd, _2nx8.nvm, _2nx8.si, _2nx8_MockRandom_0.doc, _2nx8_MockRandom_0.sd, _2nx8_MockRandom_0.tbk, _2nx8_MockRandom_0.tix, _2nx9.cfe, _2nx9.cfs, _2nx9.si, _2nxa.fdt, _2nxa.fdx, _2nxa.fnm, _2nxa.nvd, _2nxa.nvm, _2nxa.si, _2nxa_MockRandom_0.doc, _2nxa_MockRandom_0.sd, _2nxa_MockRandom_0.tio, _2nxa_MockRandom_0.tipo, _2nxb.fdt, _2nxb.fdx, _2nxb.fnm, _2nxb.nvd, _2nxb.nvm, _2nxb.si, _2nxb_MockRandom_0.doc, _2nxb_MockRandom_0.sd, _2nxb_MockRandom_0.tmp, _2nxc.fdt, _2nxc.fdx, _2nxc.fnm, _2nxc.nvd, _2nxc.nvm, _2nxc.si, _2nxc_MockRandom_0.doc, _2nxc_MockRandom_0.sd, _2nxc_MockRandom_0.tio, _2nxc_MockRandom_0.tipo, _2nxd.fdt, _2nxd.fdx, _2nxd.fnm, _2nxd.nvd, _2nxd.nvm, _2nxd.si, _2nxd_MockRandom_0.doc, _2nxd_MockRandom_0.sd, _2nxd_MockRandom_0.tim, _2nxd_MockRandom_0.tip, _2nxe.fdt, _2nxe.fdx, _2nxe.fnm, _2nxe.nvd, _2nxe.nvm, _2nxe.si, _2nxe_MockRandom_0.doc, _2nxe_MockRandom_0.sd, _2nxe_MockRandom_0.tbk, _2nxe_MockRandom_0.tix, _2nxf.fdt, _2nxf.fdx, _2nxf.fnm, _2nxf.nvd, _2nxf.nvm, _2nxf.si, _2nxf_MockRandom_0.doc, _2nxf_MockRandom_0.sd, _2nxf_MockRandom_0.tmp, _2nxh.fdt, _2nxh.fdx, _2nxh.fnm, _2nxh.nvd, _2nxh.nvm, _2nxh.si, _2nxh_MockRandom_0.doc, _2nxh_MockRandom_0.sd, _2nxh_MockRandom_0.tim, _2nxh_MockRandom_0.tip, _2nxi.fdt, _2nxi.fdx, _2nxi.fnm, _2nxi.nvd, _2nxi.nvm, _2nxi.si, _2nxi_MockRandom_0.doc, _2nxi_MockRandom_0.sd, _2nxi_MockRandom_0.tim, _2nxi_MockRandom_0.tip, _2nxj.fdt, _2nxj.fdx, _2nxj.fnm, _2nxj.nvd, _2nxj.nvm, _2nxj.si, _2nxj_MockRandom_0.doc, _2nxj_MockRandom_0.sd, _2nxj_MockRandom_0.tib, _2nxj_MockRandom_0.tii, _2nxk.fdt, _2nxk.fdx, _2nxk.fnm, _2nxk.nvd, _2nxk.nvm, _2nxk.si, _2nxk_MockRandom_0.doc, _2nxk_MockRandom_0.sd, _2nxk_MockRandom_0.tim, _2nxk_MockRandom_0.tip, segments_2]}, {indexVersion=1449825644507,generation=3,filelist=[_2nvt.fdt, _2nvt.fdx, _2nvt.fnm, _2nvt.nvd, _2nvt.nvm, _2nvt.si, _2nvt_MockRandom_0.doc, _2nvt_MockRandom_0.sd, _2nvt_MockRandom_0.tmp, _2nwu.fdt, _2nwu.fdx, _2nwu.fnm, _2nwu.nvd, _2nwu.nvm, _2nwu.si, _2nwu_MockRandom_0.doc, _2nwu_MockRandom_0.sd, _2nwu_MockRandom_0.tim, _2nwu_MockRandom_0.tip, _2nx1.fdt, _2nx1.fdx, _2nx1.fnm, _2nx1.nvd, _2nx1.nvm, _2nx1.si, _2nx1_MockRandom_0.doc, _2nx1_MockRandom_0.sd, _2nx1_MockRandom_0.tmp, _2nx2.fdt, _2nx2.fdx, _2nx2.fnm, _2nx2.nvd, _2nx2.nvm, _2nx2.si, _2nx2_MockRandom_0.doc, _2nx2_MockRandom_0.sd, _2nx2_MockRandom_0.tib, _2nx2_MockRandom_0.tiv, _2nx4.fdt, _2nx4.fdx, _2nx4.fnm, _2nx4.nvd, _2nx4.nvm, _2nx4.si, _2nx4_MockRandom_0.doc, _2nx4_MockRandom_0.sd, _2nx4_MockRandom_0.tbk, _2nx4_MockRandom_0.tix, _2nxa.fdt, _2nxa.fdx, _2nxa.fnm, _2nxa.nvd, _2nxa.nvm, _2nxa.si, _2nxa_MockRandom_0.doc, _2nxa_MockRandom_0.sd, _2nxa_MockRandom_0.tio, _2nxa_MockRandom_0.tipo, _2nxb.fdt, _2nxb.fdx, _2nxb.fnm, _2nxb.nvd, _2nxb.nvm, _2nxb.si, _2nxb_MockRandom_0.doc, _2nxb_MockRandom_0.sd, _2nxb_MockRandom_0.tmp, _2nxc.fdt, _2nxc.fdx, _2nxc.fnm, _2nxc.nvd, _2nxc.nvm, _2nxc.si, _2nxc_MockRandom_0.doc, _2nxc_MockRandom_0.sd, _2nxc_MockRandom_0.tio, _2nxc_MockRandom_0.tipo, _2nxf.fdt, _2nxf.fdx, _2nxf.fnm, _2nxf.nvd, _2nxf.nvm, _2nxf.si, _2nxf_MockRandom_0.doc, _2nxf_MockRandom_0.sd, _2nxf_MockRandom_0.tmp, _2nxg.fdt, _2nxg.fdx, _2nxg.fnm, _2nxg.nvd, _2nxg.nvm, _2nxg.si, _2nxg_MockRandom_0.doc, _2nxg_MockRandom_0.sd, _2nxg_MockRandom_0.tmp, _2nxh.fdt, _2nxh.fdx, _2nxh.fnm, _2nxh.nvd, _2nxh.nvm, _2nxh.si, _2nxh_MockRandom_0.doc, _2nxh_MockRandom_0.sd, _2nxh_MockRandom_0.tim, _2nxh_MockRandom_0.tip, _2nxi.fdt, _2nxi.fdx, _2nxi.fnm, _2nxi.nvd, _2nxi.nvm, _2nxi.si, _2nxi_MockRandom_0.doc, _2nxi_MockRandom_0.sd, _2nxi_MockRandom_0.tim, _2nxi_MockRandom_0.tip, _2nxj.fdt, _2nxj.fdx, _2nxj.fnm, _2nxj.nvd, _2nxj.nvm, _2nxj.si, _2nxj_MockRandom_0.doc, _2nxj_MockRandom_0.sd, _2nxj_MockRandom_0.tib, _2nxj_MockRandom_0.tii, _2nxk.fdt, _2nxk.fdx, _2nxk.fnm, _2nxk.nvd, _2nxk.nvm, _2nxk.si, _2nxk_MockRandom_0.doc, _2nxk_MockRandom_0.sd, _2nxk_MockRandom_0.tim, _2nxk_MockRandom_0.tip, segments_3]}]>

Stack Trace:
java.lang.AssertionError: expected:<[{indexVersion=1449825644507,generation=2,filelist=[_2njz.fdt, _2njz.fdx, _2njz.fnm, _2njz.nvd, _2njz.nvm, _2njz.si, _2njz_MockRandom_0.doc, _2njz_MockRandom_0.sd, _2njz_MockRandom_0.tbk, _2njz_MockRandom_0.tix, _2nve.cfe, _2nve.cfs, _2nve.si, _2nvh.fdt, _2nvh.fdx, _2nvh.fnm, _2nvh.nvd, _2nvh.nvm, _2nvh.si, _2nvh_MockRandom_0.doc, _2nvh_MockRandom_0.sd, _2nvh_MockRandom_0.tib, _2nvh_MockRandom_0.tii, _2nvn.cfe, _2nvn.cfs, _2nvn.si, _2nvp.fdt, _2nvp.fdx, _2nvp.fnm, _2nvp.nvd, _2nvp.nvm, _2nvp.si, _2nvp_MockRandom_0.doc, _2nvp_MockRandom_0.sd, _2nvp_MockRandom_0.tim, _2nvp_MockRandom_0.tip, _2nvq.cfe, _2nvq.cfs, _2nvq.si, _2nvr.fdt, _2nvr.fdx, _2nvr.fnm, _2nvr.nvd, _2nvr.nvm, _2nvr.si, _2nvr_MockRandom_0.doc, _2nvr_MockRandom_0.sd, _2nvr_MockRandom_0.tbk, _2nvr_MockRandom_0.tix, _2nwm.fdt, _2nwm.fdx, _2nwm.fnm, _2nwm.nvd, _2nwm.nvm, _2nwm.si, _2nwm_MockRandom_0.doc, _2nwm_MockRandom_0.sd, _2nwm_MockRandom_0.tmp, _2nwp.cfe, _2nwp.cfs, _2nwp.si, _2nwq.fdt, _2nwq.fdx, _2nwq.fnm, _2nwq.nvd, _2nwq.nvm, _2nwq.si, _2nwq_MockRandom_0.doc, _2nwq_MockRandom_0.sd, _2nwq_MockRandom_0.tbk, _2nwq_MockRandom_0.tix, _2nwr.fdt, _2nwr.fdx, _2nwr.fnm, _2nwr.nvd, _2nwr.nvm, _2nwr.si, _2nwr_MockRandom_0.doc, _2nwr_MockRandom_0.sd, _2nwr_MockRandom_0.tio, _2nwr_MockRandom_0.tipo, _2nws.fdt, _2nws.fdx, _2nws.fnm, _2nws.nvd, _2nws.nvm, _2nws.si, _2nws_MockRandom_0.doc, _2nws_MockRandom_0.sd, _2nws_MockRandom_0.tio, _2nws_MockRandom_0.tipo, _2nwu.fdt, _2nwu.fdx, _2nwu.fnm, _2nwu.nvd, _2nwu.nvm, _2nwu.si, _2nwu_MockRandom_0.doc, _2nwu_MockRandom_0.sd, _2nwu_MockRandom_0.tim, _2nwu_MockRandom_0.tip, _2nwx.fdt, _2nwx.fdx, _2nwx.fnm, _2nwx.nvd, _2nwx.nvm, _2nwx.si, _2nwx_MockRandom_0.doc, _2nwx_MockRandom_0.sd, _2nwx_MockRandom_0.tio, _2nwx_MockRandom_0.tipo, _2nx1.fdt, _2nx1.fdx, _2nx1.fnm, _2nx1.nvd, _2nx1.nvm, _2nx1.si, _2nx1_MockRandom_0.doc, _2nx1_MockRandom_0.sd, _2nx1_MockRandom_0.tmp, _2nx2.fdt, _2nx2.fdx, _2nx2.fnm, _2nx2.nvd, _2nx2.nvm, _2nx2.si, _2nx2_MockRandom_0.doc, _2nx2_MockRandom_0.sd, _2nx2_MockRandom_0.tib, _2nx2_MockRandom_0.tiv, _2nx3.cfe, _2nx3.cfs, _2nx3.si, _2nx4.fdt, _2nx4.fdx, _2nx4.fnm, _2nx4.nvd, _2nx4.nvm, _2nx4.si, _2nx4_MockRandom_0.doc, _2nx4_MockRandom_0.sd, _2nx4_MockRandom_0.tbk, _2nx4_MockRandom_0.tix, _2nx5.fdt, _2nx5.fdx, _2nx5.fnm, _2nx5.nvd, _2nx5.nvm, _2nx5.si, _2nx5_MockRandom_0.doc, _2nx5_MockRandom_0.sd, _2nx5_MockRandom_0.tmp, _2nx6.fdt, _2nx6.fdx, _2nx6.fnm, _2nx6.nvd, _2nx6.nvm, _2nx6.si, _2nx6_MockRandom_0.doc, _2nx6_MockRandom_0.sd, _2nx6_MockRandom_0.tib, _2nx6_MockRandom_0.tii, _2nx7.fdt, _2nx7.fdx, _2nx7.fnm, _2nx7.nvd, _2nx7.nvm, _2nx7.si, _2nx7_MockRandom_0.doc, _2nx7_MockRandom_0.sd, _2nx7_MockRandom_0.tbk, _2nx7_MockRandom_0.tix, _2nx8.fdt, _2nx8.fdx, _2nx8.fnm, _2nx8.nvd, _2nx8.nvm, _2nx8.si, _2nx8_MockRandom_0.doc, _2nx8_MockRandom_0.sd, _2nx8_MockRandom_0.tbk, _2nx8_MockRandom_0.tix, _2nx9.cfe, _2nx9.cfs, _2nx9.si, _2nxa.fdt, _2nxa.fdx, _2nxa.fnm, _2nxa.nvd, _2nxa.nvm, _2nxa.si, _2nxa_MockRandom_0.doc, _2nxa_MockRandom_0.sd, _2nxa_MockRandom_0.tio, _2nxa_MockRandom_0.tipo, _2nxb.fdt, _2nxb.fdx, _2nxb.fnm, _2nxb.nvd, _2nxb.nvm, _2nxb.si, _2nxb_MockRandom_0.doc, _2nxb_MockRandom_0.sd, _2nxb_MockRandom_0.tmp, _2nxc.fdt, _2nxc.fdx, _2nxc.fnm, _2nxc.nvd, _2nxc.nvm, _2nxc.si, _2nxc_MockRandom_0.doc, _2nxc_MockRandom_0.sd, _2nxc_MockRandom_0.tio, _2nxc_MockRandom_0.tipo, _2nxd.fdt, _2nxd.fdx, _2nxd.fnm, _2nxd.nvd, _2nxd.nvm, _2nxd.si, _2nxd_MockRandom_0.doc, _2nxd_MockRandom_0.sd, _2nxd_MockRandom_0.tim, _2nxd_MockRandom_0.tip, _2nxe.fdt, _2nxe.fdx, _2nxe.fnm, _2nxe.nvd, _2nxe.nvm, _2nxe.si, _2nxe_MockRandom_0.doc, _2nxe_MockRandom_0.sd, _2nxe_MockRandom_0.tbk, _2nxe_MockRandom_0.tix, _2nxf.fdt, _2nxf.fdx, _2nxf.fnm, _2nxf.nvd, _2nxf.nvm, _2nxf.si, _2nxf_MockRandom_0.doc, _2nxf_MockRandom_0.sd, _2nxf_MockRandom_0.tmp, _2nxh.fdt, _2nxh.fdx, _2nxh.fnm, _2nxh.nvd, _2nxh.nvm, _2nxh.si, _2nxh_MockRandom_0.doc, _2nxh_MockRandom_0.sd, _2nxh_MockRandom_0.tim, _2nxh_MockRandom_0.tip, _2nxi.fdt, _2nxi.fdx, _2nxi.fnm, _2nxi.nvd, _2nxi.nvm, _2nxi.si, _2nxi_MockRandom_0.doc, _2nxi_MockRandom_0.sd, _2nxi_MockRandom_0.tim, _2nxi_MockRandom_0.tip, _2nxj.fdt, _2nxj.fdx, _2nxj.fnm, _2nxj.nvd, _2nxj.nvm, _2nxj.si, _2nxj_MockRandom_0.doc, _2nxj_MockRandom_0.sd, _2nxj_MockRandom_0.tib, _2nxj_MockRandom_0.tii, _2nxk.fdt, _2nxk.fdx, _2nxk.fnm, _2nxk.nvd, _2nxk.nvm, _2nxk.si, _2nxk_MockRandom_0.doc, _2nxk_MockRandom_0.sd, _2nxk_MockRandom_0.tim, _2nxk_MockRandom_0.tip, segments_2]}]> but was:<[{indexVersion=1449825644507,generation=2,filelist=[_2njz.fdt, _2njz.fdx, _2njz.fnm, _2njz.nvd, _2njz.nvm, _2njz.si, _2njz_MockRandom_0.doc, _2njz_MockRandom_0.sd, _2njz_MockRandom_0.tbk, _2njz_MockRandom_0.tix, _2nve.cfe, _2nve.cfs, _2nve.si, _2nvh.fdt, _2nvh.fdx, _2nvh.fnm, _2nvh.nvd, _2nvh.nvm, _2nvh.si, _2nvh_MockRandom_0.doc, _2nvh_MockRandom_0.sd, _2nvh_MockRandom_0.tib, _2nvh_MockRandom_0.tii, _2nvn.cfe, _2nvn.cfs, _2nvn.si, _2nvp.fdt, _2nvp.fdx, _2nvp.fnm, _2nvp.nvd, _2nvp.nvm, _2nvp.si, _2nvp_MockRandom_0.doc, _2nvp_MockRandom_0.sd, _2nvp_MockRandom_0.tim, _2nvp_MockRandom_0.tip, _2nvq.cfe, _2nvq.cfs, _2nvq.si, _2nvr.fdt, _2nvr.fdx, _2nvr.fnm, _2nvr.nvd, _2nvr.nvm, _2nvr.si, _2nvr_MockRandom_0.doc, _2nvr_MockRandom_0.sd, _2nvr_MockRandom_0.tbk, _2nvr_MockRandom_0.tix, _2nwm.fdt, _2nwm.fdx, _2nwm.fnm, _2nwm.nvd, _2nwm.nvm, _2nwm.si, _2nwm_MockRandom_0.doc, _2nwm_MockRandom_0.sd, _2nwm_MockRandom_0.tmp, _2nwp.cfe, _2nwp.cfs, _2nwp.si, _2nwq.fdt, _2nwq.fdx, _2nwq.fnm, _2nwq.nvd, _2nwq.nvm, _2nwq.si, _2nwq_MockRandom_0.doc, _2nwq_MockRandom_0.sd, _2nwq_MockRandom_0.tbk, _2nwq_MockRandom_0.tix, _2nwr.fdt, _2nwr.fdx, _2nwr.fnm, _2nwr.nvd, _2nwr.nvm, _2nwr.si, _2nwr_MockRandom_0.doc, _2nwr_MockRandom_0.sd, _2nwr_MockRandom_0.tio, _2nwr_MockRandom_0.tipo, _2nws.fdt, _2nws.fdx, _2nws.fnm, _2nws.nvd, _2nws.nvm, _2nws.si, _2nws_MockRandom_0.doc, _2nws_MockRandom_0.sd, _2nws_MockRandom_0.tio, _2nws_MockRandom_0.tipo, _2nwu.fdt, _2nwu.fdx, _2nwu.fnm, _2nwu.nvd, _2nwu.nvm, _2nwu.si, _2nwu_MockRandom_0.doc, _2nwu_MockRandom_0.sd, _2nwu_MockRandom_0.tim, _2nwu_MockRandom_0.tip, _2nwx.fdt, _2nwx.fdx, _2nwx.fnm, _2nwx.nvd, _2nwx.nvm, _2nwx.si, _2nwx_MockRandom_0.doc, _2nwx_MockRandom_0.sd, _2nwx_MockRandom_0.tio, _2nwx_MockRandom_0.tipo, _2nx1.fdt, _2nx1.fdx, _2nx1.fnm, _2nx1.nvd, _2nx1.nvm, _2nx1.si, _2nx1_MockRandom_0.doc, _2nx1_MockRandom_0.sd, _2nx1_MockRandom_0.tmp, _2nx2.fdt, _2nx2.fdx, _2nx2.fnm, _2nx2.nvd, _2nx2.nvm, _2nx2.si, _2nx2_MockRandom_0.doc, _2nx2_MockRandom_0.sd, _2nx2_MockRandom_0.tib, _2nx2_MockRandom_0.tiv, _2nx3.cfe, _2nx3.cfs, _2nx3.si, _2nx4.fdt, _2nx4.fdx, _2nx4.fnm, _2nx4.nvd, _2nx4.nvm, _2nx4.si, _2nx4_MockRandom_0.doc, _2nx4_MockRandom_0.sd, _2nx4_MockRandom_0.tbk, _2nx4_MockRandom_0.tix, _2nx5.fdt, _2nx5.fdx, _2nx5.fnm, _2nx5.nvd, _2nx5.nvm, _2nx5.si, _2nx5_MockRandom_0.doc, _2nx5_MockRandom_0.sd, _2nx5_MockRandom_0.tmp, _2nx6.fdt, _2nx6.fdx, _2nx6.fnm, _2nx6.nvd, _2nx6.nvm, _2nx6.si, _2nx6_MockRandom_0.doc, _2nx6_MockRandom_0.sd, _2nx6_MockRandom_0.tib, _2nx6_MockRandom_0.tii, _2nx7.fdt, _2nx7.fdx, _2nx7.fnm, _2nx7.nvd, _2nx7.nvm, _2nx7.si, _2nx7_MockRandom_0.doc, _2nx7_MockRandom_0.sd, _2nx7_MockRandom_0.tbk, _2nx7_MockRandom_0.tix, _2nx8.fdt, _2nx8.fdx, _2nx8.fnm, _2nx8.nvd, _2nx8.nvm, _2nx8.si, _2nx8_MockRandom_0.doc, _2nx8_MockRandom_0.sd, _2nx8_MockRandom_0.tbk, _2nx8_MockRandom_0.tix, _2nx9.cfe, _2nx9.cfs, _2nx9.si, _2nxa.fdt, _2nxa.fdx, _2nxa.fnm, _2nxa.nvd, _2nxa.nvm, _2nxa.si, _2nxa_MockRandom_0.doc, _2nxa_MockRandom_0.sd, _2nxa_MockRandom_0.tio, _2nxa_MockRandom_0.tipo, _2nxb.fdt, _2nxb.fdx, _2nxb.fnm, _2nxb.nvd, _2nxb.nvm, _2nxb.si, _2nxb_MockRandom_0.doc, _2nxb_MockRandom_0.sd, _2nxb_MockRandom_0.tmp, _2nxc.fdt, _2nxc.fdx, _2nxc.fnm, _2nxc.nvd, _2nxc.nvm, _2nxc.si, _2nxc_MockRandom_0.doc, _2nxc_MockRandom_0.sd, _2nxc_MockRandom_0.tio, _2nxc_MockRandom_0.tipo, _2nxd.fdt, _2nxd.fdx, _2nxd.fnm, _2nxd.nvd, _2nxd.nvm, _2nxd.si, _2nxd_MockRandom_0.doc, _2nxd_MockRandom_0.sd, _2nxd_MockRandom_0.tim, _2nxd_MockRandom_0.tip, _2nxe.fdt, _2nxe.fdx, _2nxe.fnm, _2nxe.nvd, _2nxe.nvm, _2nxe.si, _2nxe_MockRandom_0.doc, _2nxe_MockRandom_0.sd, _2nxe_MockRandom_0.tbk, _2nxe_MockRandom_0.tix, _2nxf.fdt, _2nxf.fdx, _2nxf.fnm, _2nxf.nvd, _2nxf.nvm, _2nxf.si, _2nxf_MockRandom_0.doc, _2nxf_MockRandom_0.sd, _2nxf_MockRandom_0.tmp, _2nxh.fdt, _2nxh.fdx, _2nxh.fnm, _2nxh.nvd, _2nxh.nvm, _2nxh.si, _2nxh_MockRandom_0.doc, _2nxh_MockRandom_0.sd, _2nxh_MockRandom_0.tim, _2nxh_MockRandom_0.tip, _2nxi.fdt, _2nxi.fdx, _2nxi.fnm, _2nxi.nvd, _2nxi.nvm, _2nxi.si, _2nxi_MockRandom_0.doc, _2nxi_MockRandom_0.sd, _2nxi_MockRandom_0.tim, _2nxi_MockRandom_0.tip, _2nxj.fdt, _2nxj.fdx, _2nxj.fnm, _2nxj.nvd, _2nxj.nvm, _2nxj.si, _2nxj_MockRandom_0.doc, _2nxj_MockRandom_0.sd, _2nxj_MockRandom_0.tib, _2nxj_MockRandom_0.tii, _2nxk.fdt, _2nxk.fdx, _2nxk.fnm, _2nxk.nvd, _2nxk.nvm, _2nxk.si, _2nxk_MockRandom_0.doc, _2nxk_MockRandom_0.sd, _2nxk_MockRandom_0.tim, _2nxk_MockRandom_0.tip, segments_2]}, {indexVersion=1449825644507,generation=3,filelist=[_2nvt.fdt, _2nvt.fdx, _2nvt.fnm, _2nvt.nvd, _2nvt.nvm, _2nvt.si, _2nvt_MockRandom_0.doc, _2nvt_MockRandom_0.sd, _2nvt_MockRandom_0.tmp, _2nwu.fdt, _2nwu.fdx, _2nwu.fnm, _2nwu.nvd, _2nwu.nvm, _2nwu.si, _2nwu_MockRandom_0.doc, _2nwu_MockRandom_0.sd, _2nwu_MockRandom_0.tim, _2nwu_MockRandom_0.tip, _2nx1.fdt, _2nx1.fdx, _2nx1.fnm, _2nx1.nvd, _2nx1.nvm, _2nx1.si, _2nx1_MockRandom_0.doc, _2nx1_MockRandom_0.sd, _2nx1_MockRandom_0.tmp, _2nx2.fdt, _2nx2.fdx, _2nx2.fnm, _2nx2.nvd, _2nx2.nvm, _2nx2.si, _2nx2_MockRandom_0.doc, _2nx2_MockRandom_0.sd, _2nx2_MockRandom_0.tib, _2nx2_MockRandom_0.tiv, _2nx4.fdt, _2nx4.fdx, _2nx4.fnm, _2nx4.nvd, _2nx4.nvm, _2nx4.si, _2nx4_MockRandom_0.doc, _2nx4_MockRandom_0.sd, _2nx4_MockRandom_0.tbk, _2nx4_MockRandom_0.tix, _2nxa.fdt, _2nxa.fdx, _2nxa.fnm, _2nxa.nvd, _2nxa.nvm, _2nxa.si, _2nxa_MockRandom_0.doc, _2nxa_MockRandom_0.sd, _2nxa_MockRandom_0.tio, _2nxa_MockRandom_0.tipo, _2nxb.fdt, _2nxb.fdx, _2nxb.fnm, _2nxb.nvd, _2nxb.nvm, _2nxb.si, _2nxb_MockRandom_0.doc, _2nxb_MockRandom_0.sd, _2nxb_MockRandom_0.tmp, _2nxc.fdt, _2nxc.fdx, _2nxc.fnm, _2nxc.nvd, _2nxc.nvm, _2nxc.si, _2nxc_MockRandom_0.doc, _2nxc_MockRandom_0.sd, _2nxc_MockRandom_0.tio, _2nxc_MockRandom_0.tipo, _2nxf.fdt, _2nxf.fdx, _2nxf.fnm, _2nxf.nvd, _2nxf.nvm, _2nxf.si, _2nxf_MockRandom_0.doc, _2nxf_MockRandom_0.sd, _2nxf_MockRandom_0.tmp, _2nxg.fdt, _2nxg.fdx, _2nxg.fnm, _2nxg.nvd, _2nxg.nvm, _2nxg.si, _2nxg_MockRandom_0.doc, _2nxg_MockRandom_0.sd, _2nxg_MockRandom_0.tmp, _2nxh.fdt, _2nxh.fdx, _2nxh.fnm, _2nxh.nvd, _2nxh.nvm, _2nxh.si, _2nxh_MockRandom_0.doc, _2nxh_MockRandom_0.sd, _2nxh_MockRandom_0.tim, _2nxh_MockRandom_0.tip, _2nxi.fdt, _2nxi.fdx, _2nxi.fnm, _2nxi.nvd, _2nxi.nvm, _2nxi.si, _2nxi_MockRandom_0.doc, _2nxi_MockRandom_0.sd, _2nxi_MockRandom_0.tim, _2nxi_MockRandom_0.tip, _2nxj.fdt, _2nxj.fdx, _2nxj.fnm, _2nxj.nvd, _2nxj.nvm, _2nxj.si, _2nxj_MockRandom_0.doc, _2nxj_MockRandom_0.sd, _2nxj_MockRandom_0.tib, _2nxj_MockRandom_0.tii, _2nxk.fdt, _2nxk.fdx, _2nxk.fnm, _2nxk.nvd, _2nxk.nvm, _2nxk.si, _2nxk_MockRandom_0.doc, _2nxk_MockRandom_0.sd, _2nxk_MockRandom_0.tim, _2nxk_MockRandom_0.tip, segments_3]}]>
	at __randomizedtesting.SeedInfo.seed([252FCDCEA534147F:F8D6FED57C1A7C]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.failNotEquals(Assert.java:647)
	at org.junit.Assert.assertEquals(Assert.java:128)
	at org.junit.Assert.assertEquals(Assert.java:147)
	at org.apache.solr.handler.TestReplicationHandler.doTestReplicateAfterCoreReload(TestReplicationHandler.java:1140)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.handler.TestReplicationHandler.doTestStressReplication

Error Message:
timed out waiting for collection1 startAt time to exceed: Fri Dec 11 22:22:02 GMT+13:00 2015

Stack Trace:
java.lang.AssertionError: timed out waiting for collection1 startAt time to exceed: Fri Dec 11 22:22:02 GMT+13:00 2015
	at __randomizedtesting.SeedInfo.seed([252FCDCEA534147F:FE84CD08A01C7DCC]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.solr.handler.TestReplicationHandler.watchCoreStartAt(TestReplicationHandler.java:1419)
	at org.apache.solr.handler.TestReplicationHandler.doTestStressReplication(TestReplicationHandler.java:771)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 10887 lines...]
   [junit4] Suite: org.apache.solr.handler.TestReplicationHandler
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_252FCDCEA534147F-001/init-core-data-001
   [junit4]   2> 1107487 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigReplication-seed#[252FCDCEA534147F]) [    ] o.a.s.SolrTestCaseJ4 ###Starting doTestIndexAndConfigReplication
   [junit4]   2> 1107487 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigReplication-seed#[252FCDCEA534147F]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_252FCDCEA534147F-001/solr-instance-001/collection1
   [junit4]   2> 1107491 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigReplication-seed#[252FCDCEA534147F]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 1107493 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigReplication-seed#[252FCDCEA534147F]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@16de7d5f{/solr,null,AVAILABLE}
   [junit4]   2> 1107494 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigReplication-seed#[252FCDCEA534147F]) [    ] o.e.j.s.ServerConnector Started ServerConnector@68f8f3b3{HTTP/1.1}{127.0.0.1:35053}
   [junit4]   2> 1107494 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigReplication-seed#[252FCDCEA534147F]) [    ] o.e.j.s.Server Started @1110494ms
   [junit4]   2> 1107494 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigReplication-seed#[252FCDCEA534147F]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostPort=35053, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_252FCDCEA534147F-001/solr-instance-001/collection1/data, hostContext=/solr}
   [junit4]   2> 1107494 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigReplication-seed#[252FCDCEA534147F]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@7b3cb2c6
   [junit4]   2> 1107494 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigReplication-seed#[252FCDCEA534147F]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_252FCDCEA534147F-001/solr-instance-001'
   [junit4]   2> 1107494 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigReplication-seed#[252FCDCEA534147F]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1107495 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigReplication-seed#[252FCDCEA534147F]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1107495 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigReplication-seed#[252FCDCEA534147F]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_252FCDCEA534147F-001/solr-instance-001/solr.xml
   [junit4]   2> 1107510 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigReplication-seed#[252FCDCEA534147F]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_252FCDCEA534147F-001/solr-instance-001/.
   [junit4]   2> 1107510 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigReplication-seed#[252FCDCEA534147F]) [    ] o.a.s.c.CoreContainer New CoreContainer 1347987264
   [junit4]   2> 1107510 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigReplication-seed#[252FCDCEA534147F]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_252FCDCEA534147F-001/solr-instance-001]
   [junit4]   2> 1107510 WARN  (TEST-TestReplicationHandler.doTestIndexAndConfigReplication-seed#[252FCDCEA534147F]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_252FCDCEA534147F-001/solr-instance-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_252FCDCEA534147F-001/solr-instance-001/lib
   [junit4]   2> 1107510 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigReplication-seed#[252FCDCEA534147F]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1107511 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigReplication-seed#[252FCDCEA534147F]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1107512 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigReplication-seed#[252FCDCEA534147F]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1107512 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigReplication-seed#[252FCDCEA534147F]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1107512 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigReplication-seed#[252FCDCEA534147F]) [    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1107512 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigReplication-seed#[252FCDCEA534147F]) [    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1107513 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigReplication-seed#[252FCDCEA534147F]) [    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_252FCDCEA534147F-001/solr-instance-001/.
   [junit4]   2> 1107514 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigReplication-seed#[252FCDCEA534147F]) [    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {transient=false, loadOnStartup=true, config=solrconfig.xml, configSetProperties=configsetprops.json, name=collection1, dataDir=data/, schema=schema.xml}
   [junit4]   2> 1107514 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigReplication-seed#[252FCDCEA534147F]) [    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_252FCDCEA534147F-001/solr-instance-001/./collection1
   [junit4]   2> 1107514 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigReplication-seed#[252FCDCEA534147F]) [    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 1107515 INFO  (coreLoadExecutor-2297-thread-1) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_252FCDCEA534147F-001/solr-instance-001/collection1'
   [junit4]   2> 1107515 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigReplication-seed#[252FCDCEA534147F]) [    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0
   [junit4]   2> 1107515 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigReplication-seed#[252FCDCEA534147F]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1107515 INFO  (coreLoadExecutor-2297-thread-1) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1107515 INFO  (coreLoadExecutor-2297-thread-1) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1107523 INFO  (coreLoadExecutor-2297-thread-1) [    ] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 1107536 INFO  (coreLoadExecutor-2297-thread-1) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.5.0
   [junit4]   2> 1107561 INFO  (coreLoadExecutor-2297-thread-1) [    ] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1107565 INFO  (coreLoadExecutor-2297-thread-1) [    ] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1107573 INFO  (coreLoadExecutor-2297-thread-1) [    ] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 1107575 INFO  (coreLoadExecutor-2297-thread-1) [    ] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_252FCDCEA534147F-001/solr-instance-001/collection1'
   [junit4]   2> 1107575 INFO  (coreLoadExecutor-2297-thread-1) [    ] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_252FCDCEA534147F-001/solr-instance-001/./collection1
   [junit4]   2> 1107575 INFO  (coreLoadExecutor-2297-thread-1) [    x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 1107575 INFO  (coreLoadExecutor-2297-thread-1) [    x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_252FCDCEA534147F-001/solr-instance-001/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_252FCDCEA534147F-001/solr-instance-001/./collection1/data/]
   [junit4]   2> 1107575 INFO  (coreLoadExecutor-2297-thread-1) [    x:collection1] o.a.s.c.SolrCore JMX monitoring not detected for core: collection1
   [junit4]   2> 1107575 INFO  (coreLoadExecutor-2297-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_252FCDCEA534147F-001/solr-instance-001/./collection1/data
   [junit4]   2> 1107575 INFO  (coreLoadExecutor-2297-thread-1) [    x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_252FCDCEA534147F-001/solr-instance-001/./collection1/data/index/
   [junit4]   2> 1107576 WARN  (coreLoadExecutor-2297-thread-1) [    x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_252FCDCEA534147F-001/solr-instance-001/./collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 1107576 INFO  (coreLoadExecutor-2297-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_252FCDCEA534147F-001/solr-instance-001/./collection1/data/index
   [junit4]   2> 1107576 INFO  (coreLoadExecutor-2297-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=1187930328, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
   [junit4]   2> 1107577 INFO  (coreLoadExecutor-2297-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@49765f6 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@6fa6fbd4),segFN=segments_1,generation=1}
   [junit4]   2> 1107577 INFO  (coreLoadExecutor-2297-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 1107577 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [    x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_252FCDCEA534147F-001/solr-instance-001/./collection1/data/
   [junit4]   2> 1107578 INFO  (coreLoadExecutor-2297-thread-1) [    x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 1107581 INFO  (coreLoadExecutor-2297-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1107583 INFO  (coreLoadExecutor-2297-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1107584 INFO  (coreLoadExecutor-2297-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1107585 INFO  (coreLoadExecutor-2297-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1107601 INFO  (coreLoadExecutor-2297-thread-1) [    x:collection1] o.a.s.c.PluginBag requestHandler : 'lazy' created with startup=lazy 
   [junit4]   2> 1107601 INFO  (coreLoadExecutor-2297-thread-1) [    x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/threads,/schema,/update/json/docs,/admin/system,standard,lazy,/admin/logging,/admin/properties,/update/json,/admin/plugins,/get,/replication,/admin/file,/admin/mbeans,defaults,/admin/segments,/update/csv,/update,/admin/luke,/config,/admin/ping
   [junit4]   2> 1107602 INFO  (coreLoadExecutor-2297-thread-1) [    x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 1107603 INFO  (coreLoadExecutor-2297-thread-1) [    x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1107603 INFO  (coreLoadExecutor-2297-thread-1) [    x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1107603 INFO  (coreLoadExecutor-2297-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@40455297
   [junit4]   2> 1107603 INFO  (coreLoadExecutor-2297-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@49765f6 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@6fa6fbd4),segFN=segments_1,generation=1}
   [junit4]   2> 1107603 INFO  (coreLoadExecutor-2297-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 1107603 INFO  (coreLoadExecutor-2297-thread-1) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@37bf255[collection1] main
   [junit4]   2> 1107604 INFO  (coreLoadExecutor-2297-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_252FCDCEA534147F-001/solr-instance-001/collection1/conf
   [junit4]   2> 1107604 INFO  (coreLoadExecutor-2297-thread-1) [    x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {storageDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_252FCDCEA534147F-001/solr-instance-001/collection1/conf}
   [junit4]   2> 1107604 INFO  (coreLoadExecutor-2297-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using file:dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_252FCDCEA534147F-001/solr-instance-001/collection1/conf
   [junit4]   2> 1107604 INFO  (coreLoadExecutor-2297-thread-1) [    x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 1107604 INFO  (coreLoadExecutor-2297-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler Replication enabled for following config files: schema.xml,xslt/dummy.xsl
   [junit4]   2> 1107604 INFO  (coreLoadExecutor-2297-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1107604 INFO  (coreLoadExecutor-2297-thread-1) [    x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 1107604 INFO  (searcherExecutor-2298-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@37bf255[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1108016 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigReplication-seed#[252FCDCEA534147F]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_252FCDCEA534147F-001/solr-instance-002/collection1
   [junit4]   2> 1108020 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigReplication-seed#[252FCDCEA534147F]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 1108022 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigReplication-seed#[252FCDCEA534147F]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@62dcc762{/solr,null,AVAILABLE}
   [junit4]   2> 1108022 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigReplication-seed#[252FCDCEA534147F]) [    ] o.e.j.s.ServerConnector Started ServerConnector@6ff0f6d1{HTTP/1.1}{127.0.0.1:46287}
   [junit4]   2> 1108022 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigReplication-seed#[252FCDCEA534147F]) [    ] o.e.j.s.Server Started @1111022ms
   [junit4]   2> 1108022 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigReplication-seed#[252FCDCEA534147F]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_252FCDCEA534147F-001/solr-instance-002/collection1/data, hostContext=/solr, hostPort=46287}
   [junit4]   2> 1108023 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigReplication-seed#[252FCDCEA534147F]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@7b3cb2c6
   [junit4]   2> 1108023 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigReplication-seed#[252FCDCEA534147F]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_252FCDCEA534147F-001/solr-instance-002'
   [junit4]   2> 1108023 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigReplication-seed#[252FCDCEA534147F]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1108023 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigReplication-seed#[252FCDCEA534147F]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1108023 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigReplication-seed#[252FCDCEA534147F]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_252FCDCEA534147F-001/solr-instance-002/solr.xml
   [junit4]   2> 1108037 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigReplication-seed#[252FCDCEA534147F]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_252FCDCEA534147F-001/solr-instance-002/.
   [junit4]   2> 1108037 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigReplication-seed#[252FCDCEA534147F]) [    ] o.a.s.c.CoreContainer New CoreContainer 974364157
   [junit4]   2> 1108037 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigReplication-seed#[252FCDCEA534147F]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_252FCDCEA534147F-001/solr-instance-002]
   [junit4]   2> 1108037 WARN  (TEST-TestReplicationHandler.doTestIndexAndConfigReplication-seed#[252FCDCEA534147F]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_252FCDCEA534147F-001/solr-instance-002/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_252FCDCEA534147F-001/solr-instance-002/lib
   [junit4]   2> 1108037 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigReplication-seed#[252FCDCEA534147F]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1108039 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigReplication-seed#[252FCDCEA534147F]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1108039 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigReplication-seed#[252FCDCEA534147F]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1108039 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigReplication-seed#[252FCDCEA534147F]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1108039 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigReplication-seed#[252FCDCEA534147F]) [    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1108039 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigReplication-seed#[252FCDCEA534147F]) [    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1108040 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigReplication-seed#[252FCDCEA534147F]) [    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_252FCDCEA534147F-001/solr-instance-002/.
   [junit4]   2> 1108041 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigReplication-seed#[252FCDCEA534147F]) [    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {transient=false, dataDir=data/, config=solrconfig.xml, loadOnStartup=true, configSetProperties=configsetprops.json, name=collection1, schema=schema.xml}
   [junit4]   2> 1108041 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigReplication-seed#[252FCDCEA534147F]) [    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_252FCDCEA534147F-001/solr-instance-002/./collection1
   [junit4]   2> 1108041 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigReplication-seed#[252FCDCEA534147F]) [    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 1108042 INFO  (coreLoadExecutor-2307-thread-1) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_252FCDCEA534147F-001/solr-instance-002/collection1'
   [junit4]   2> 1108042 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigReplication-seed#[252FCDCEA534147F]) [    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0
   [junit4]   2> 1108042 INFO  (coreLoadExecutor-2307-thread-1) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1108042 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigReplication-seed#[252FCDCEA534147F]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1108042 INFO  (coreLoadExecutor-2307-thread-1) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1108050 INFO  (coreLoadExecutor-2307-thread-1) [    ] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 1108063 INFO  (coreLoadExecutor-2307-thread-1) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.5.0
   [junit4]   2> 1108089 INFO  (coreLoadExecutor-2307-thread-1) [    ] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1108094 INFO  (coreLoadExecutor-2307-thread-1) [    ] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1108101 INFO  (coreLoadExecutor-2307-thread-1) [    ] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 1108103 INFO  (coreLoadExecutor-2307-thread-1) [    ] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_252FCDCEA534147F-001/solr-instance-002/collection1'
   [junit4]   2> 1108104 INFO  (coreLoadExecutor-2307-thread-1) [    ] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_252FCDCEA534147F-001/solr-instance-002/./collection1
   [junit4]   2> 1108104 INFO  (coreLoadExecutor-2307-thread-1) [    x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 1108104 INFO  (coreLoadExecutor-2307-thread-1) [    x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_252FCDCEA534147F-001/solr-instance-002/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_252FCDCEA534147F-001/solr-instance-002/./collection1/data/]
   [junit4]   2> 1108104 INFO  (coreLoadExecutor-2307-thread-1) [    x:collection1] o.a.s.c.SolrCore JMX monitoring not detected for core: collection1
   [junit4]   2> 1108104 INFO  (coreLoadExecutor-2307-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_252FCDCEA534147F-001/solr-instance-002/./collection1/data
   [junit4]   2> 1108104 INFO  (coreLoadExecutor-2307-thread-1) [    x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_252FCDCEA534147F-001/solr-instance-002/./collection1/data/index/
   [junit4]   2> 1108105 WARN  (coreLoadExecutor-2307-thread-1) [    x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_252FCDCEA534147F-001/solr-instance-002/./collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 1108105 INFO  (coreLoadExecutor-2307-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_252FCDCEA534147F-001/solr-instance-002/./collection1/data/index
   [junit4]   2> 1108105 INFO  (coreLoadExecutor-2307-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=1187930328, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
   [junit4]   2> 1108106 INFO  (coreLoadExecutor-2307-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@2a08ff5b lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@34b4c4ec),segFN=segments_1,generation=1}
   [junit4]   2> 1108106 INFO  (coreLoadExecutor-2307-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 1108106 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [    x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_252FCDCEA534147F-001/solr-instance-002/./collection1/data/
   [junit4]   2> 1108108 INFO  (coreLoadExecutor-2307-thread-1) [    x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 1108111 INFO  (coreLoadExecutor-2307-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1108112 INFO  (coreLoadExecutor-2307-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1108113 INFO  (coreLoadExecutor-2307-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1108114 INFO  (coreLoadExecutor-2307-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1108116 INFO  (coreLoadExecutor-2307-thread-1) [    x:collection1] o.a.s.c.PluginBag requestHandler : 'lazy' created with startup=lazy 
   [junit4]   2> 1108116 INFO  (coreLoadExecutor-2307-thread-1) [    x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/threads,/schema,/update/json/docs,/admin/system,standard,lazy,/admin/logging,/admin/properties,/update/json,/admin/plugins,/get,/replication,/admin/file,/admin/mbeans,defaults,/admin/segments,/update/csv,/update,/admin/luke,/config,/admin/ping
   [junit4]   2> 1108117 INFO  (coreLoadExecutor-2307-thread-1) [    x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 1108118 INFO  (coreLoadExecutor-2307-thread-1) [    x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1108118 INFO  (coreLoadExecutor-2307-thread-1) [    x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1108118 INFO  (coreLoadExecutor-2307-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@7155e71f
   [junit4]   2> 1108118 INFO  (coreLoadExecutor-2307-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@2a08ff5b lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@34b4c4ec),segFN=segments_1,generation=1}
   [junit4]   2> 1108118 INFO  (coreLoadExecutor-2307-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 1108118 INFO  (coreLoadExecutor-2307-thread-1) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@41d39a0e[collection1] main
   [junit4]   2> 1108118 INFO  (coreLoadExecutor-2307-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_252FCDCEA534147F-001/solr-instance-002/collection1/conf
   [junit4]   2> 1108119 INFO  (coreLoadExecutor-2307-thread-1) [    x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {storageDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_252FCDCEA534147F-001/solr-instance-002/collection1/conf}
   [junit4]   2> 1108119 INFO  (coreLoadExecutor-2307-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using file:dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_252FCDCEA534147F-001/solr-instance-002/collection1/conf
   [junit4]   2> 1108119 INFO  (coreLoadExecutor-2307-thread-1) [    x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 1108121 INFO  (coreLoadExecutor-2307-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler Poll scheduled at an interval of 1000ms
   [junit4]   2> 1108121 INFO  (coreLoadExecutor-2307-thread-1) [    x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 1108121 INFO  (searcherExecutor-2308-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@41d39a0e[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1108124 INFO  (qtp351150450-56791) [    x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={wt=javabin&qt=/replication&command=indexversion&version=2} status=0 QTime=0 
   [junit4]   2> 1108545 INFO  (qtp351150450-56792) [    x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/select params={wt=javabin&version=2&sort=id+desc&q=*:*} hits=0 status=0 QTime=0 
   [junit4]   2> 1108549 INFO  (qtp351150450-56790) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[0]} 0 1
   [junit4]   2> 1108553 INFO  (qtp351150450-56795) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[1]} 0 1
   [junit4]   2> 1108556 INFO  (qtp351150450-56789) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[2]} 0 1
   [junit4]   2> 1108559 INFO  (qtp351150450-56796) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[3]} 0 1
   [junit4]   2> 1108562 INFO  (qtp351150450-56791) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[4]} 0 0
   [junit4]   2> 1108566 INFO  (qtp351150450-56792) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[5]} 0 1
   [junit4]   2> 1108568 INFO  (qtp351150450-56790) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[6]} 0 0
   [junit4]   2> 1108597 INFO  (qtp351150450-56795) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[7]} 0 1
   [junit4]   2> 1108599 INFO  (qtp351150450-56789) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[8]} 0 0
   [junit4]   2> 1108602 INFO  (qtp351150450-56796) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[9]} 0 1
   [junit4]   2> 1108605 INFO  (qtp351150450-56791) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[10]} 0 0
   [junit4]   2> 1108607 INFO  (qtp351150450-56792) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[11]} 0 1
   [junit4]   2> 1108610 INFO  (qtp351150450-56790) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[12]} 0 0
   [junit4]   2> 1108613 INFO  (qtp351150450-56795) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[13]} 0 1
   [junit4]   2> 1108615 INFO  (qtp351150450-56789) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[14]} 0 0
   [junit4]   2> 1108618 INFO  (qtp351150450-56796) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[15]} 0 1
   [junit4]   2> 1108620 INFO  (qtp351150450-56791) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[16]} 0 0
   [junit4]   2> 1108623 INFO  (qtp351150450-56792) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[17]} 0 1
   [junit4]   2> 1108626 INFO  (qtp351150450-56790) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[18]} 0 0
   [junit4]   2> 1108629 INFO  (qtp351150450-56795) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[19]} 0 1
   [junit4]   2> 1108631 INFO  (qtp351150450-56789) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[20]} 0 0
   [junit4]   2> 1108634 INFO  (qtp351150450-56796) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[21]} 0 1
   [junit4]   2> 1108637 INFO  (qtp351150450-56791) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[22]} 0 0
   [junit4]   2> 1108640 INFO  (qtp351150450-56792) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[23]} 0 1
   [junit4]   2> 1108643 INFO  (qtp351150450-56790) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[24]} 0 0
   [junit4]   2> 1108645 INFO  (qtp351150450-56795) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[25]} 0 1
   [junit4]   2> 1108647 INFO  (qtp351150450-56789) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[26]} 0 0
   [junit4]   2> 1108650 INFO  (qtp351150450-56796) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[27]} 0 1
   [junit4]   2> 1108652 INFO  (qtp351150450-56791) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[28]} 0 0
   [junit4]   2> 1108655 INFO  (qtp351150450-56792) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[29]} 0 1
   [junit4]   2> 1108657 INFO  (qtp351150450-56790) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[30]} 0 0
   [junit4]   2> 1108660 INFO  (qtp351150450-56795) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[31]} 0 1
   [junit4]   2> 1108662 INFO  (qtp351150450-56789) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[32]} 0 0
   [junit4]   2> 1108665 INFO  (qtp351150450-56796) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[33]} 0 1
   [junit4]   2> 1108667 INFO  (qtp351150450-56791) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[34]} 0 0
   [junit4]   2> 1108670 INFO  (qtp351150450-56792) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[35]} 0 1
   [junit4]   2> 1108672 INFO  (qtp351150450-56790) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[36]} 0 0
   [junit4]   2> 1108675 INFO  (qtp351150450-56795) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[37]} 0 1
   [junit4]   2> 1108677 INFO  (qtp351150450-56789) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[38]} 0 0
   [junit4]   2> 1108680 INFO  (qtp351150450-56796) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[39]} 0 1
   [junit4]   2> 1108682 INFO  (qtp351150450-56791) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[40]} 0 0
   [junit4]   2> 1108685 INFO  (qtp351150450-56792) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[41]} 0 1
   [junit4]   2> 1108687 INFO  (qtp351150450-56790) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[42]} 0 0
   [junit4]   2> 1108695 INFO  (qtp351150450-56795) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[43]} 0 5
   [junit4]   2> 1108697 INFO  (qtp351150450-56789) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[44]} 0 0
   [junit4]   2> 1108700 INFO  (qtp351150450-56796) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[45]} 0 1
   [junit4]   2> 1108702 INFO  (qtp351150450-56791) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[46]} 0 0
   [junit4]   2> 1108705 INFO  (qtp351150450-56792) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[47]} 0 1
   [junit4]   2> 1108708 INFO  (qtp351150450-56790) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[48]} 0 0
   [junit4]   2> 1108711 INFO  (qtp351150450-56795) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[49]} 0 1
   [junit4]   2> 1108713 INFO  (qtp351150450-56789) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[50]} 0 0
   [junit4]   2> 1108717 INFO  (qtp351150450-56796) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[51]} 0 1
   [junit4]   2> 1108719 INFO  (qtp351150450-56791) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[52]} 0 0
   [junit4]   2> 1108722 INFO  (qtp351150450-56792) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[53]} 0 1
   [junit4]   2> 1108724 INFO  (qtp351150450-56790) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[54]} 0 0
   [junit4]   2> 1108727 INFO  (qtp351150450-56795) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[55]} 0 1
   [junit4]   2> 1108729 INFO  (qtp351150450-56789) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[56]} 0 0
   [junit4]   2> 1108732 INFO  (qtp351150450-56796) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[57]} 0 1
   [junit4]   2> 1108734 INFO  (qtp351150450-56791) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[58]} 0 0
   [junit4]   2> 1108738 INFO  (qtp351150450-56792) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[59]} 0 1
   [junit4]   2> 1108740 INFO  (qtp351150450-56790) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[60]} 0 0
   [junit4]   2> 1108803 INFO  (qtp351150450-56795) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[61]} 0 61
   [junit4]   2> 1108806 INFO  (qtp351150450-56789) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[62]} 0 0
   [junit4]   2> 1108812 INFO  (qtp351150450-56796) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[63]} 0 4
   [junit4]   2> 1108814 INFO  (qtp351150450-56791) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[64]} 0 0
   [junit4]   2> 1108818 INFO  (qtp351150450-56792) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[65]} 0 2
   [junit4]   2> 1108820 INFO  (qtp351150450-56790) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[66]} 0 0
   [junit4]   2> 1108824 INFO  (qtp351150450-56795) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[67]} 0 1
   [junit4]   2> 1108826 INFO  (qtp351150450-56789) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[68]} 0 0
   [junit4]   2> 1108829 INFO  (qtp351150450-56796) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[69]} 0 1
   [junit4]   2> 1108831 INFO  (qtp351150450-56791) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[70]} 0 0
   [junit4]   2> 1108837 INFO  (qtp351150450-56792) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[71]} 0 4
   [junit4]   2> 1108840 INFO  (qtp351150450-56790) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[72]} 0 0
   [junit4]   2> 1108842 INFO  (qtp351150450-56795) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[73]} 0 1
   [junit4]   2> 1108845 INFO  (qtp351150450-56789) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[74]} 0 0
   [junit4]   2> 1108851 INFO  (qtp351150450-56796) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[75]} 0 3
   [junit4]   2> 1108853 INFO  (qtp351150450-56791) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[76]} 0 0
   [junit4]   2> 1108856 INFO  (qtp351150450-56792) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[77]} 0 1
   [junit4]   2> 1108858 INFO  (qtp351150450-56790) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[78]} 0 0
   [junit4]   2> 1108861 INFO  (qtp351150450-56795) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[79]} 0 1
   [junit4]   2> 1108863 INFO  (qtp351150450-56789) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[80]} 0 0
   [junit4]   2> 1108866 INFO  (qtp351150450-56796) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[81]} 0 1
   [junit4]   2> 1108868 INFO  (qtp351150450-56791) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[82]} 0 0
   [junit4]   2> 1108871 INFO  (qtp351150450-56792) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[83]} 0 1
   [junit4]   2> 1108873 INFO  (qtp351150450-56790) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[84]} 0 0
   [junit4]   2> 1108875 INFO  (qtp351150450-56795) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[85]} 0 1
   [junit4]   2> 1108877 INFO  (qtp351150450-56789) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[86]} 0 0
   [junit4]   2> 1108880 INFO  (qtp351150450-56796) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[87]} 0 1
   [junit4]   2> 1108882 INFO  (qtp351150450-56791) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[88]} 0 0
   [junit4]   2> 1108885 INFO  (qtp351150450-56792) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[89]} 0 1
   [junit4]   2> 1108887 INFO  (qtp351150450-56790) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[90]} 0 0
   [junit4]   2> 1108890 INFO  (qtp351150450-56795) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[91]} 0 1
   [junit4]   2> 1108893 INFO  (qtp351150450-56789) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[92]} 0 0
   [junit4]   2> 1108895 INFO  (qtp351150450-56796) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[93]} 0 1
   [junit4]   2> 1108897 INFO  (qtp351150450-56791) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[94]} 0 0
   [junit4]   2> 1108900 INFO  (qtp351150450-56792) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[95]} 0 1
   [junit4]   2> 1108902 INFO  (qtp351150450-56790) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[96]} 0 0
   [junit4]   2> 1108905 INFO  (qtp351150450-56795) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[97]} 0 1
   [junit4]   2> 1108907 INFO  (qtp351150450-56789) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[98]} 0 0
   [junit4]   2> 1108910 INFO  (qtp351150450-56796) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[99]} 0 1
   [junit4]   2> 1108912 INFO  (qtp351150450-56791) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[100]} 0 0
   [junit4]   2> 1108916 INFO  (qtp351150450-56792) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[101]} 0 1
   [junit4]   2> 1108919 INFO  (qtp351150450-56790) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[102]} 0 1
   [junit4]   2> 1108929 INFO  (qtp351150450-56795) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[103]} 0 8
   [junit4]   2> 1108931 INFO  (qtp351150450-56789) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[104]} 0 0
   [junit4]   2> 1108936 INFO  (qtp351150450-56796) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[105]} 0 3
   [junit4]   2> 1108939 INFO  (qtp351150450-56791) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[106]} 0 0
   [junit4]   2> 1108943 INFO  (qtp351150450-56792) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[107]} 0 2
   [junit4]   2> 1108994 INFO  (qtp351150450-56790) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[108]} 0 49
   [junit4]   2> 1108997 INFO  (qtp351150450-56795) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[109]} 0 1
   [junit4]   2> 1108999 INFO  (qtp351150450-56789) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[110]} 0 0
   [junit4]   2> 1109002 INFO  (qtp351150450-56796) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[111]} 0 1
   [junit4]   2> 1109004 INFO  (qtp351150450-56791) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[112]} 0 0
   [junit4]   2> 1109007 INFO  (qtp351150450-56792) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[113]} 0 1
   [junit4]   2> 1109009 INFO  (qtp351150450-56790) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[114]} 0 0
   [junit4]   2> 1109012 INFO  (qtp351150450-56795) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[115]} 0 0
   [junit4]   2> 1109014 INFO  (qtp351150450-56789) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[116]} 0 0
   [junit4]   2> 1109017 INFO  (qtp351150450-56796) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[117]} 0 0
   [junit4]   2> 1109019 INFO  (qtp351150450-56791) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[118]} 0 0
   [junit4]   2> 1109021 INFO  (qtp351150450-56792) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[119]} 0 1
   [junit4]   2> 1109024 INFO  (qtp351150450-56790) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [c

[...truncated too long message...]

dler.doTestIndexAndConfigAliasReplication-seed#[252FCDCEA534147F]) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_252FCDCEA534147F-001/solr-instance-027/./collection1/data
   [junit4]   2> 2123296 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[252FCDCEA534147F]) [    x:collection1] o.a.s.c.CachingDirectoryFactory looking to close /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_252FCDCEA534147F-001/solr-instance-027/./collection1/data/index [CachedDir<<refCount=0;path=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_252FCDCEA534147F-001/solr-instance-027/./collection1/data/index;done=false>>]
   [junit4]   2> 2123296 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[252FCDCEA534147F]) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_252FCDCEA534147F-001/solr-instance-027/./collection1/data/index
   [junit4]   2> 2123296 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[252FCDCEA534147F]) [    x:collection1] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@5f966a00{/solr,null,UNAVAILABLE}
   [junit4]   2> 2123297 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[252FCDCEA534147F]) [    ] o.e.j.s.ServerConnector Stopped ServerConnector@20a391fd{HTTP/1.1}{127.0.0.1:0}
   [junit4]   2> 2123297 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[252FCDCEA534147F]) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1069535444
   [junit4]   2> 2123298 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[252FCDCEA534147F]) [    x:collection1] o.a.s.c.SolrCore [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@39384d4e
   [junit4]   2> 2123298 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[252FCDCEA534147F]) [    x:collection1] o.a.s.u.DirectUpdateHandler2 closing DirectUpdateHandler2{commits=1,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=1,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
   [junit4]   2> 2123298 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[252FCDCEA534147F]) [    x:collection1] o.a.s.u.SolrCoreState Closing SolrCoreState
   [junit4]   2> 2123299 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[252FCDCEA534147F]) [    x:collection1] o.a.s.u.DefaultSolrCoreState SolrCoreState ref count has reached 0 - closing IndexWriter
   [junit4]   2> 2123299 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[252FCDCEA534147F]) [    x:collection1] o.a.s.u.DefaultSolrCoreState closing IndexWriter with IndexWriterCloser
   [junit4]   2> 2123299 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[252FCDCEA534147F]) [    x:collection1] o.a.s.c.SolrCore [collection1] Closing main searcher on request.
   [junit4]   2> 2123299 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[252FCDCEA534147F]) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing MockDirectoryFactory - 2 directories currently being tracked
   [junit4]   2> 2123299 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[252FCDCEA534147F]) [    x:collection1] o.a.s.c.CachingDirectoryFactory looking to close /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_252FCDCEA534147F-001/solr-instance-028/./collection1/data/index.20151211222255571 [CachedDir<<refCount=0;path=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_252FCDCEA534147F-001/solr-instance-028/./collection1/data/index.20151211222255571;done=false>>]
   [junit4]   2> 2123299 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[252FCDCEA534147F]) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_252FCDCEA534147F-001/solr-instance-028/./collection1/data/index.20151211222255571
   [junit4]   2> 2123299 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[252FCDCEA534147F]) [    x:collection1] o.a.s.c.CachingDirectoryFactory looking to close /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_252FCDCEA534147F-001/solr-instance-028/./collection1/data [CachedDir<<refCount=0;path=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_252FCDCEA534147F-001/solr-instance-028/./collection1/data;done=false>>]
   [junit4]   2> 2123299 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[252FCDCEA534147F]) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_252FCDCEA534147F-001/solr-instance-028/./collection1/data
   [junit4]   2> 2123300 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[252FCDCEA534147F]) [    x:collection1] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@b548fe8{/solr,null,UNAVAILABLE}
   [junit4]   2> 2123301 INFO  (SUITE-TestReplicationHandler-seed#[252FCDCEA534147F]-worker) [    ] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_252FCDCEA534147F-001
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene54): {newname=PostingsFormat(name=MockRandom), name=PostingsFormat(name=MockRandom), id=PostingsFormat(name=MockRandom)}, docValues:{}, sim=RandomSimilarityProvider(queryNorm=false,coord=yes): {}, locale=fr, timezone=Etc/GMT-13
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_80 (64-bit)/cpus=4,threads=1,free=149649120,total=530055168
   [junit4]   2> NOTE: All tests run in this JVM: [TestSolr4Spatial, BasicAuthIntegrationTest, TestValueSourceCache, TestRestManager, TestRandomDVFaceting, TestDocBasedVersionConstraints, UUIDFieldTest, TestSortingResponseWriter, DistributedFacetPivotWhiteBoxTest, DefaultValueUpdateProcessorTest, TestShardHandlerFactory, RankQueryTest, HLLUtilTest, TestRTGBase, TestPKIAuthenticationPlugin, TestSolrQueryParserResource, TestBlendedInfixSuggestions, IndexSchemaTest, VersionInfoTest, TestManagedSchema, RAMDirectoryFactoryTest, BigEndianAscendingWordDeserializerTest, TestCSVLoader, TestNoOpRegenerator, ReturnFieldsTest, TestShortCircuitedRequests, TestMiniSolrCloudCluster, TestFaceting, TestStressLucene, TestHighFrequencyDictionaryFactory, TestRealTimeGet, ResponseHeaderTest, UUIDUpdateProcessorFallbackTest, CopyFieldTest, TestBinaryField, TestFoldingMultitermQuery, TestDistributedSearch, TestReplicaProperties, DistributedSuggestComponentTest, ResponseLogComponentTest, CloudExitableDirectoryReaderTest, StandardRequestHandlerTest, OverseerTaskQueueTest, DeleteShardTest, JsonLoaderTest, DocumentAnalysisRequestHandlerTest, SynonymTokenizerTest, TestLRUCache, OutputWriterTest, TestManagedSynonymFilterFactory, DistributedSpellCheckComponentTest, OverseerCollectionConfigSetProcessorTest, TestJsonRequest, DateFieldTest, TestDocSet, CacheHeaderTest, TestAtomicUpdateErrorCases, PreAnalyzedFieldTest, EchoParamsTest, DisMaxRequestHandlerTest, HdfsChaosMonkeyNothingIsSafeTest, HdfsWriteToMultipleCollectionsTest, AddBlockUpdateTest, ParsingFieldUpdateProcessorsTest, AnalyticsQueryTest, CollectionTooManyReplicasTest, TestStressUserVersions, HdfsDirectoryFactoryTest, DeleteInactiveReplicaTest, InfoHandlerTest, TestReplicationHandler]
   [junit4] Completed [217/554 (1!)] on J0 in 1015.84s, 13 tests, 2 failures <<< FAILURES!

[...truncated 1071 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:801: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:738: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1456: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1010: There were test failures: 554 suites (7 ignored), 2188 tests, 2 failures, 33 ignored (18 assumptions) [seed: 252FCDCEA534147F]

Total time: 215 minutes 18 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1042 - Still Failing

[Apache Jenkins Server <je...@builds.apache.org>]

Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1042/

1 tests failed.
FAILED:  org.apache.lucene.search.TestGeoPointQuery.testRandomBig

Error Message:
GC overhead limit exceeded

Stack Trace:
java.lang.OutOfMemoryError: GC overhead limit exceeded
	at __randomizedtesting.SeedInfo.seed([5DB3A1C37B327030:DAE4DC4CEA6B0CB0]:0)
	at org.apache.lucene.codecs.memory.DirectPostingsFormat$DirectField.<init>(DirectPostingsFormat.java:427)
	at org.apache.lucene.codecs.memory.DirectPostingsFormat$DirectFields.<init>(DirectPostingsFormat.java:131)
	at org.apache.lucene.codecs.memory.DirectPostingsFormat.fieldsProducer(DirectPostingsFormat.java:115)
	at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsReader.<init>(PerFieldPostingsFormat.java:261)
	at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat.fieldsProducer(PerFieldPostingsFormat.java:341)
	at org.apache.lucene.index.SegmentCoreReaders.<init>(SegmentCoreReaders.java:104)
	at org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:65)
	at org.apache.lucene.index.ReadersAndUpdates.getReader(ReadersAndUpdates.java:145)
	at org.apache.lucene.index.BufferedUpdatesStream$SegmentState.<init>(BufferedUpdatesStream.java:384)
	at org.apache.lucene.index.BufferedUpdatesStream.openSegmentStates(BufferedUpdatesStream.java:416)
	at org.apache.lucene.index.BufferedUpdatesStream.applyDeletesAndUpdates(BufferedUpdatesStream.java:261)
	at org.apache.lucene.index.IndexWriter.applyAllDeletesAndUpdates(IndexWriter.java:3161)
	at org.apache.lucene.index.IndexWriter.maybeApplyDeletes(IndexWriter.java:3147)
	at org.apache.lucene.index.IndexWriter.doFlush(IndexWriter.java:3124)
	at org.apache.lucene.index.IndexWriter.flush(IndexWriter.java:3087)
	at org.apache.lucene.index.IndexWriter.forceMerge(IndexWriter.java:1741)
	at org.apache.lucene.index.IndexWriter.forceMerge(IndexWriter.java:1721)
	at org.apache.lucene.util.BaseGeoPointTestCase.verify(BaseGeoPointTestCase.java:596)
	at org.apache.lucene.util.BaseGeoPointTestCase.doTestRandom(BaseGeoPointTestCase.java:399)
	at org.apache.lucene.util.BaseGeoPointTestCase.testRandomBig(BaseGeoPointTestCase.java:327)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)




Build Log:
[...truncated 8481 lines...]
   [junit4] Suite: org.apache.lucene.search.TestGeoPointQuery
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestGeoPointQuery -Dtests.method=testRandomBig -Dtests.seed=5DB3A1C37B327030 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=el -Dtests.timezone=Etc/Universal -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
   [junit4] ERROR    126s J1 | TestGeoPointQuery.testRandomBig <<<
   [junit4]    > Throwable #1: java.lang.OutOfMemoryError: GC overhead limit exceeded
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([5DB3A1C37B327030:DAE4DC4CEA6B0CB0]:0)
   [junit4]    > 	at org.apache.lucene.codecs.memory.DirectPostingsFormat$DirectField.<init>(DirectPostingsFormat.java:427)
   [junit4]    > 	at org.apache.lucene.codecs.memory.DirectPostingsFormat$DirectFields.<init>(DirectPostingsFormat.java:131)
   [junit4]    > 	at org.apache.lucene.codecs.memory.DirectPostingsFormat.fieldsProducer(DirectPostingsFormat.java:115)
   [junit4]    > 	at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsReader.<init>(PerFieldPostingsFormat.java:261)
   [junit4]    > 	at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat.fieldsProducer(PerFieldPostingsFormat.java:341)
   [junit4]    > 	at org.apache.lucene.index.SegmentCoreReaders.<init>(SegmentCoreReaders.java:104)
   [junit4]    > 	at org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:65)
   [junit4]    > 	at org.apache.lucene.index.ReadersAndUpdates.getReader(ReadersAndUpdates.java:145)
   [junit4]    > 	at org.apache.lucene.index.BufferedUpdatesStream$SegmentState.<init>(BufferedUpdatesStream.java:384)
   [junit4]    > 	at org.apache.lucene.index.BufferedUpdatesStream.openSegmentStates(BufferedUpdatesStream.java:416)
   [junit4]    > 	at org.apache.lucene.index.BufferedUpdatesStream.applyDeletesAndUpdates(BufferedUpdatesStream.java:261)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.applyAllDeletesAndUpdates(IndexWriter.java:3161)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.maybeApplyDeletes(IndexWriter.java:3147)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.doFlush(IndexWriter.java:3124)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.flush(IndexWriter.java:3087)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.forceMerge(IndexWriter.java:1741)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.forceMerge(IndexWriter.java:1721)
   [junit4]    > 	at org.apache.lucene.util.BaseGeoPointTestCase.verify(BaseGeoPointTestCase.java:596)
   [junit4]    > 	at org.apache.lucene.util.BaseGeoPointTestCase.doTestRandom(BaseGeoPointTestCase.java:399)
   [junit4]    > 	at org.apache.lucene.util.BaseGeoPointTestCase.testRandomBig(BaseGeoPointTestCase.java:327)
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/sandbox/test/J1/temp/lucene.search.TestGeoPointQuery_5DB3A1C37B327030-001
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene54): {point=PostingsFormat(name=Direct), id=PostingsFormat(name=Memory doPackFST= false), string=Lucene50(blocksize=128)}, docValues:{point=DocValuesFormat(name=Asserting), id=DocValuesFormat(name=Memory)}, sim=DefaultSimilarity, locale=el, timezone=Etc/Universal
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_80 (64-bit)/cpus=4,threads=1,free=196115464,total=403701760
   [junit4]   2> NOTE: All tests run in this JVM: [DuplicateFilterTest, TestGeoPointQuery]
   [junit4] Completed [18/20 (1!)] on J1 in 386.20s, 23 tests, 1 error <<< FAILURES!

[...truncated 4 lines...]
   [junit4] JVM J1: stdout was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/sandbox/test/temp/junit4-J1-20151210_164328_858.sysout
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] java.lang.OutOfMemoryError: GC overhead limit exceeded
   [junit4] Dumping heap to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/heapdumps/java_pid18413.hprof ...
   [junit4] Heap dump file created [552835499 bytes in 5.683 secs]
   [junit4] <<< JVM J1: EOF ----

[...truncated 13 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:801: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:738: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build.xml:472: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:2252: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/module-build.xml:58: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1456: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1010: There were test failures: 20 suites, 142 tests, 1 error [seed: 5DB3A1C37B327030]

Total time: 116 minutes 36 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1041 - Still Failing

[Apache Jenkins Server <je...@builds.apache.org>]

Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1041/

1 tests failed.
FAILED:  org.apache.solr.cloud.hdfs.HdfsBasicDistributedZk2Test.test

Error Message:


Stack Trace:
java.lang.NullPointerException
	at __randomizedtesting.SeedInfo.seed([BF6DB73668EA9BBA:373988ECC616F642]:0)
	at org.apache.solr.client.solrj.embedded.JettySolrRunner.getCoreContainer(JettySolrRunner.java:374)
	at org.apache.solr.cloud.ChaosMonkey.expireSession(ChaosMonkey.java:124)
	at org.apache.solr.cloud.BasicDistributedZk2Test.test(BasicDistributedZk2Test.java:131)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:965)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:940)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 10273 lines...]
   [junit4] Suite: org.apache.solr.cloud.hdfs.HdfsBasicDistributedZk2Test
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_BF6DB73668EA9BBA-001/init-core-data-001
   [junit4]   2> 703861 INFO  (SUITE-HdfsBasicDistributedZk2Test-seed#[BF6DB73668EA9BBA]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   1> Formatting using clusterid: testClusterID
   [junit4]   2> 703950 WARN  (SUITE-HdfsBasicDistributedZk2Test-seed#[BF6DB73668EA9BBA]-worker) [    ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
   [junit4]   2> 703966 WARN  (SUITE-HdfsBasicDistributedZk2Test-seed#[BF6DB73668EA9BBA]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 703977 INFO  (SUITE-HdfsBasicDistributedZk2Test-seed#[BF6DB73668EA9BBA]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 703992 INFO  (SUITE-HdfsBasicDistributedZk2Test-seed#[BF6DB73668EA9BBA]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/hdfs to ./temp/Jetty_localhost_41510_hdfs____.y4dasf/webapp
   [junit4]   2> 704148 INFO  (SUITE-HdfsBasicDistributedZk2Test-seed#[BF6DB73668EA9BBA]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 704475 INFO  (SUITE-HdfsBasicDistributedZk2Test-seed#[BF6DB73668EA9BBA]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:41510
   [junit4]   2> 704620 WARN  (SUITE-HdfsBasicDistributedZk2Test-seed#[BF6DB73668EA9BBA]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 704622 INFO  (SUITE-HdfsBasicDistributedZk2Test-seed#[BF6DB73668EA9BBA]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 704640 INFO  (SUITE-HdfsBasicDistributedZk2Test-seed#[BF6DB73668EA9BBA]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_40855_datanode____4gus3n/webapp
   [junit4]   2> 704832 INFO  (SUITE-HdfsBasicDistributedZk2Test-seed#[BF6DB73668EA9BBA]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 705128 INFO  (SUITE-HdfsBasicDistributedZk2Test-seed#[BF6DB73668EA9BBA]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:40855
   [junit4]   2> 705246 WARN  (SUITE-HdfsBasicDistributedZk2Test-seed#[BF6DB73668EA9BBA]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 705251 INFO  (SUITE-HdfsBasicDistributedZk2Test-seed#[BF6DB73668EA9BBA]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 705265 INFO  (SUITE-HdfsBasicDistributedZk2Test-seed#[BF6DB73668EA9BBA]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_59784_datanode____.n9srii/webapp
   [junit4]   2> 705465 INFO  (IPC Server handler 3 on 58696) [    ] BlockStateChange BLOCK* processReport: from storage DS-13b02519-4d94-4f73-a6d3-618de83dacac node DatanodeRegistration(127.0.0.1, datanodeUuid=ca1780d1-b226-41a3-926a-10724401377c, infoPort=40855, ipcPort=36581, storageInfo=lv=-56;cid=testClusterID;nsid=911232124;c=0), blocks: 0, hasStaleStorages: true, processing time: 0 msecs
   [junit4]   2> 705465 INFO  (IPC Server handler 3 on 58696) [    ] BlockStateChange BLOCK* processReport: from storage DS-434609a3-3092-4032-8859-72cd30a2283f node DatanodeRegistration(127.0.0.1, datanodeUuid=ca1780d1-b226-41a3-926a-10724401377c, infoPort=40855, ipcPort=36581, storageInfo=lv=-56;cid=testClusterID;nsid=911232124;c=0), blocks: 0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 705492 INFO  (SUITE-HdfsBasicDistributedZk2Test-seed#[BF6DB73668EA9BBA]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 705688 INFO  (SUITE-HdfsBasicDistributedZk2Test-seed#[BF6DB73668EA9BBA]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:59784
   [junit4]   2> 705900 INFO  (IPC Server handler 5 on 58696) [    ] BlockStateChange BLOCK* processReport: from storage DS-aa26c54e-7058-44a3-bfd9-8d922be4a1d2 node DatanodeRegistration(127.0.0.1, datanodeUuid=6300e33a-b4ca-4d3b-8167-df5a8e970f8f, infoPort=59784, ipcPort=37865, storageInfo=lv=-56;cid=testClusterID;nsid=911232124;c=0), blocks: 0, hasStaleStorages: true, processing time: 1 msecs
   [junit4]   2> 705900 INFO  (IPC Server handler 5 on 58696) [    ] BlockStateChange BLOCK* processReport: from storage DS-b9f7b83a-4810-4bba-bd5c-bfba988f8972 node DatanodeRegistration(127.0.0.1, datanodeUuid=6300e33a-b4ca-4d3b-8167-df5a8e970f8f, infoPort=59784, ipcPort=37865, storageInfo=lv=-56;cid=testClusterID;nsid=911232124;c=0), blocks: 0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 706082 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 706084 INFO  (Thread-3618) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 706084 INFO  (Thread-3618) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 706184 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.ZkTestServer start zk server on port:34302
   [junit4]   2> 706184 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 706184 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 706192 INFO  (zkCallback-1573-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6911a79 name:ZooKeeperConnection Watcher:127.0.0.1:34302 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 706192 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 706192 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 706192 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 706200 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 706210 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 706210 INFO  (zkCallback-1574-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@56846c0b name:ZooKeeperConnection Watcher:127.0.0.1:34302/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 706210 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 706211 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 706211 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 706214 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 706216 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 706217 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 706219 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 706219 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 706221 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 706221 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 706223 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 706223 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 706224 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 706224 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 706226 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 706226 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 706227 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 706227 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 706229 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 706229 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 706230 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 706230 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 706232 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 706232 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 706234 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 706234 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 706235 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 706236 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 706608 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_BF6DB73668EA9BBA-001/control-001/cores/collection1
   [junit4]   2> 706610 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 706618 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@3a93d624{/,null,AVAILABLE}
   [junit4]   2> 706618 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.e.j.s.ServerConnector Started ServerConnector@6d7d44cb{HTTP/1.1}{127.0.0.1:46663}
   [junit4]   2> 706618 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.e.j.s.Server Started @709456ms
   [junit4]   2> 706619 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_BF6DB73668EA9BBA-001/control-001/cores, solr.data.dir=hdfs://localhost:58696/hdfs__localhost_58696__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-5.x_solr_build_solr-core_test_J1_temp_solr.cloud.hdfs.HdfsBasicDistributedZk2Test_BF6DB73668EA9BBA-001_tempDir-002_control_data, hostPort=46663, hostContext=/}
   [junit4]   2> 706619 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@7b3cb2c6
   [junit4]   2> 706619 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_BF6DB73668EA9BBA-001/control-001'
   [junit4]   2> 706619 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 706619 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 706619 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 706620 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 706628 INFO  (zkCallback-1575-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7dd6c24 name:ZooKeeperConnection Watcher:127.0.0.1:34302/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 706629 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 706629 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 706632 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x15185de05b70002, likely client has closed socket
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 706632 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 706632 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_BF6DB73668EA9BBA-001/control-001/solr.xml
   [junit4]   2> 706653 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_BF6DB73668EA9BBA-001/control-001/cores
   [junit4]   2> 706653 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.CoreContainer New CoreContainer 134746427
   [junit4]   2> 706653 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_BF6DB73668EA9BBA-001/control-001]
   [junit4]   2> 706653 WARN  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_BF6DB73668EA9BBA-001/control-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_BF6DB73668EA9BBA-001/control-001/lib
   [junit4]   2> 706654 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 706655 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 706656 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 706656 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 706656 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:34302/solr
   [junit4]   2> 706656 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 706656 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 706658 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 706658 INFO  (zkCallback-1577-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@37708dc5 name:ZooKeeperConnection Watcher:127.0.0.1:34302 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 706660 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 706660 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 706664 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [n:127.0.0.1:46663_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 706680 INFO  (zkCallback-1578-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@50d1b0da name:ZooKeeperConnection Watcher:127.0.0.1:34302/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 706680 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [n:127.0.0.1:46663_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 706682 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [n:127.0.0.1:46663_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 706684 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [n:127.0.0.1:46663_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 706687 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [n:127.0.0.1:46663_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 706689 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [n:127.0.0.1:46663_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 706691 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [n:127.0.0.1:46663_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 706696 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [n:127.0.0.1:46663_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 706701 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [n:127.0.0.1:46663_    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 706704 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [n:127.0.0.1:46663_    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 706706 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [n:127.0.0.1:46663_    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 706711 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [n:127.0.0.1:46663_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:46663_
   [junit4]   2> 706711 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [n:127.0.0.1:46663_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:46663_
   [junit4]   2> 706714 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [n:127.0.0.1:46663_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 706718 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [n:127.0.0.1:46663_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 706720 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [n:127.0.0.1:46663_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 706720 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [n:127.0.0.1:46663_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/95004255776604164-127.0.0.1:46663_-n_0000000000
   [junit4]   2> 706721 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [n:127.0.0.1:46663_    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:46663_
   [junit4]   2> 706721 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [n:127.0.0.1:46663_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 706722 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [n:127.0.0.1:46663_    ] o.a.s.c.Overseer Overseer (id=95004255776604164-127.0.0.1:46663_-n_0000000000) starting
   [junit4]   2> 706727 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [n:127.0.0.1:46663_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 706740 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [n:127.0.0.1:46663_    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 706742 INFO  (OverseerCollectionConfigSetProcessor-95004255776604164-127.0.0.1:46663_-n_0000000000) [n:127.0.0.1:46663_    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 706744 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [n:127.0.0.1:46663_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 706744 INFO  (OverseerStateUpdate-95004255776604164-127.0.0.1:46663_-n_0000000000) [n:127.0.0.1:46663_    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 706779 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [n:127.0.0.1:46663_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 706779 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [n:127.0.0.1:46663_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 706781 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [n:127.0.0.1:46663_    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_BF6DB73668EA9BBA-001/control-001/cores
   [junit4]   2> 706782 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [n:127.0.0.1:46663_    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {shard=, configSetProperties=configsetprops.json, collection=control_collection, loadOnStartup=true, dataDir=data/, config=solrconfig.xml, transient=false, schema=schema.xml, coreNodeName=, name=collection1}
   [junit4]   2> 706782 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [n:127.0.0.1:46663_    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_BF6DB73668EA9BBA-001/control-001/cores/collection1
   [junit4]   2> 706782 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [n:127.0.0.1:46663_    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 706789 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [n:127.0.0.1:46663_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 706789 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [n:127.0.0.1:46663_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 706789 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 706789 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 706793 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 706794 INFO  (OverseerStateUpdate-95004255776604164-127.0.0.1:46663_-n_0000000000) [n:127.0.0.1:46663_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "node_name":"127.0.0.1:46663_",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:46663"} current state version: 0
   [junit4]   2> 706794 INFO  (OverseerStateUpdate-95004255776604164-127.0.0.1:46663_-n_0000000000) [n:127.0.0.1:46663_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "node_name":"127.0.0.1:46663_",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:46663"}
   [junit4]   2> 706795 INFO  (OverseerStateUpdate-95004255776604164-127.0.0.1:46663_-n_0000000000) [n:127.0.0.1:46663_    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 706795 INFO  (OverseerStateUpdate-95004255776604164-127.0.0.1:46663_-n_0000000000) [n:127.0.0.1:46663_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 706798 INFO  (zkCallback-1578-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 707794 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 707795 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 707796 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 707796 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 707797 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 707797 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_BF6DB73668EA9BBA-001/control-001/cores/collection1'
   [junit4]   2> 707797 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 707797 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 707807 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 707817 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 707830 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.5.0
   [junit4]   2> 707863 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 707878 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 708284 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 708296 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 708298 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 708322 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 708327 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 708330 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 708331 WARN  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 708331 WARN  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 708331 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 708332 WARN  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 708332 WARN  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 708333 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 708333 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 708333 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.HdfsDirectoryFactory
   [junit4]   2> 708344 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:58696/solr_hdfs_home
   [junit4]   2> 708344 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
   [junit4]   2> 708344 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_BF6DB73668EA9BBA-001/control-001/cores/collection1], dataDir=[null]
   [junit4]   2> 708345 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@64c3d9c0
   [junit4]   2> 708345 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:58696/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 708434 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:58696/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 708449 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=hdfs://localhost:58696/solr_hdfs_home/control_collection/core_node1/data/index/
   [junit4]   2> 708477 WARN  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory 'hdfs:/localhost:58696/solr_hdfs_home/control_collection/core_node1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 708477 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:58696/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 708505 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
   [junit4]   2> 708506 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
   [junit4]   2> 708506 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Creating new global HDFS BlockCache
   [junit4]   2> 708559 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
   [junit4]   2> 708559 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:58696/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 708559 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=45, maxMergeAtOnceExplicit=14, maxMergedSegmentMB=1.5556640625, floorSegmentMB=2.083984375, forceMergeDeletesPctAllowed=2.632039295550377, segmentsPerTier=45.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
   [junit4]   2> 708615 INFO  (IPC Server handler 5 on 58696) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:36944 is added to blk_1073741825_1001{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-13b02519-4d94-4f73-a6d3-618de83dacac:NORMAL:127.0.0.1:42534|RBW], ReplicaUnderConstruction[[DISK]DS-aa26c54e-7058-44a3-bfd9-8d922be4a1d2:NORMAL:127.0.0.1:36944|RBW]]} size 0
   [junit4]   2> 708619 INFO  (IPC Server handler 6 on 58696) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:42534 is added to blk_1073741825_1001{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-aa26c54e-7058-44a3-bfd9-8d922be4a1d2:NORMAL:127.0.0.1:36944|RBW], ReplicaUnderConstruction[[DISK]DS-434609a3-3092-4032-8859-72cd30a2283f:NORMAL:127.0.0.1:42534|FINALIZED]]} size 0
   [junit4]   2> 708632 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(BlockDirectory(HdfsDirectory@hdfs://localhost:58696/solr_hdfs_home/control_collection/core_node1/data/index lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@71de19d8); maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 708632 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 708635 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in hdfs://localhost:58696/solr_hdfs_home/control_collection/core_node1/data/
   [junit4]   2> 708639 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 708640 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 708640 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 708640 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 708641 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 708641 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 708641 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 708641 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 708641 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 708641 WARN  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
   [junit4]   2> 708644 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 708645 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 708655 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 708663 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 708670 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/threads,/admin/system,/admin/properties,/get,/update/csv,/admin/logging,/admin/file,/update/json,/admin/plugins,standard,/update/json/docs,/admin/segments,/admin/ping,/update,/schema,/admin/mbeans,/replication,/config,/admin/luke,/dump
   [junit4]   2> 708672 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 708678 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
   [junit4]   2> 708679 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 tlogDfsReplication=2
   [junit4]   2> 708702 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 708702 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 708704 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=32, maxMergeAtOnceExplicit=50, maxMergedSegmentMB=32.3037109375, floorSegmentMB=0.3125, forceMergeDeletesPctAllowed=12.00496103389468, segmentsPerTier=32.0, maxCFSSegmentSizeMB=1.078125, noCFSRatio=1.0
   [junit4]   2> 708810 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(BlockDirectory(HdfsDirectory@hdfs://localhost:58696/solr_hdfs_home/control_collection/core_node1/data/index lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@71de19d8); maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 708810 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 708811 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@f541914[collection1] main
   [junit4]   2> 708811 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 708813 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 708813 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 708813 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 708813 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 708813 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 708814 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 708814 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 708814 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 708817 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 708820 INFO  (searcherExecutor-4046-thread-1-processing-n:127.0.0.1:46663_ s:shard1 x:collection1 r:core_node1 c:control_collection) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@f541914[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 708822 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 708822 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 256 version buckets from index
   [junit4]   2> 708822 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 708822 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1520068095287230464
   [junit4]   2> 708822 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 0.0ms to seed version buckets with highest version 1520068095287230464
   [junit4]   2> 708822 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 708823 INFO  (coreLoadExecutor-4045-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 708824 INFO  (coreZkRegister-4039-thread-1-processing-n:127.0.0.1:46663_ s:shard1 x:collection1 r:core_node1 c:control_collection) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:46663 collection:control_collection shard:shard1
   [junit4]   2> 708828 INFO  (coreZkRegister-4039-thread-1-processing-n:127.0.0.1:46663_ s:shard1 x:collection1 r:core_node1 c:control_collection) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 708833 INFO  (coreZkRegister-4039-thread-1-processing-n:127.0.0.1:46663_ s:shard1 x:collection1 r:core_node1 c:control_collection) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/95004255776604164-core_node1-n_0000000000
   [junit4]   2> 708834 INFO  (coreZkRegister-4039-thread-1-processing-n:127.0.0.1:46663_ s:shard1 x:collection1 r:core_node1 c:control_collection) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 708836 INFO  (coreZkRegister-4039-thread-1-processing-n:127.0.0.1:46663_ s:shard1 x:collection1 r:core_node1 c:control_collection) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 708836 INFO  (coreZkRegister-4039-thread-1-processing-n:127.0.0.1:46663_ s:shard1 x:collection1 r:core_node1 c:control_collection) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 708836 INFO  (coreZkRegister-4039-thread-1-processing-n:127.0.0.1:46663_ s:shard1 x:collection1 r:core_node1 c:control_collection) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:46663/collection1/
   [junit4]   2> 708836 INFO  (coreZkRegister-4039-thread-1-processing-n:127.0.0.1:46663_ s:shard1 x:collection1 r:core_node1 c:control_collection) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 708836 INFO  (coreZkRegister-4039-thread-1-processing-n:127.0.0.1:46663_ s:shard1 x:collection1 r:core_node1 c:control_collection) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:46663/collection1/ has no replicas
   [junit4]   2> 708837 INFO  (OverseerStateUpdate-95004255776604164-127.0.0.1:46663_-n_0000000000) [n:127.0.0.1:46663_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 708838 INFO  (coreZkRegister-4039-thread-1-processing-n:127.0.0.1:46663_ s:shard1 x:collection1 r:core_node1 c:control_collection) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 708839 INFO  (zkCallback-1578-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 708844 INFO  (coreZkRegister-4039-thread-1-processing-n:127.0.0.1:46663_ s:shard1 x:collection1 r:core_node1 c:control_collection) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 708846 INFO  (coreZkRegister-4039-thread-1-processing-n:127.0.0.1:46663_ s:shard1 x:collection1 r:core_node1 c:control_collection) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:46663/collection1/ shard1
   [junit4]   2> 708848 INFO  (OverseerStateUpdate-95004255776604164-127.0.0.1:46663_-n_0000000000) [n:127.0.0.1:46663_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:46663",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 2
   [junit4]   2> 708951 INFO  (zkCallback-1578-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 708998 INFO  (coreZkRegister-4039-thread-1-processing-n:127.0.0.1:46663_ s:shard1 x:collection1 r:core_node1 c:control_collection) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:46663/collection1/ and leader is http://127.0.0.1:46663/collection1/
   [junit4]   2> 708999 INFO  (coreZkRegister-4039-thread-1-processing-n:127.0.0.1:46663_ s:shard1 x:collection1 r:core_node1 c:control_collection) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:46663
   [junit4]   2> 708999 INFO  (coreZkRegister-4039-thread-1-processing-n:127.0.0.1:46663_ s:shard1 x:collection1 r:core_node1 c:control_collection) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 708999 INFO  (coreZkRegister-4039-thread-1-processing-n:127.0.0.1:46663_ s:shard1 x:collection1 r:core_node1 c:control_collection) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 708999 INFO  (coreZkRegister-4039-thread-1-processing-n:127.0.0.1:46663_ s:shard1 x:collection1 r:core_node1 c:control_collection) [n:127.0.0.1:46663_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 709004 INFO  (OverseerStateUpdate-95004255776604164-127.0.0.1:46663_-n_0000000000) [n:127.0.0.1:46663_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "node_name":"127.0.0.1:46663_",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:46663"} current state version: 3
   [junit4]   2> 709005 INFO  (OverseerStateUpdate-95004255776604164-127.0.0.1:46663_-n_0000000000) [n:127.0.0.1:46663_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "node_name":"127.0.0.1:46663_",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:46663"}
   [junit4]   2> 709111 INFO  (zkCallback-1578-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 709292 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 709293 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 709294 INFO  (zkCallback-1580-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2f0fa4ae name:ZooKeeperConnection Watcher:127.0.0.1:34302/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 709294 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 709294 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 709294 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 709298 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 709298 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with stateFormat=2
   [junit4]   2> 709298 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 709298 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 709300 INFO  (zkCallback-1581-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@347edc46 name:ZooKeeperConnection Watcher:127.0.0.1:34302/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 709300 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 709301 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 709306 INFO  (OverseerStateUpdate-95004255776604164-127.0.0.1:46663_-n_0000000000) [n:127.0.0.1:46663_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"create",
   [junit4]   2>   "name":"collection1",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "stateFormat":"2"} current state version: 4
   [junit4]   2> 709306 INFO  (OverseerStateUpdate-95004255776604164-127.0.0.1:46663_-n_0000000000) [n:127.0.0.1:46663_    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 709422 INFO  (OverseerStateUpdate-95004255776604164-127.0.0.1:46663_-n_0000000000) [n:127.0.0.1:46663_    ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/collection1/state.json
   [junit4]   2> 709652 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_BF6DB73668EA9BBA-001/shard-1-001/cores/collection1
   [junit4]   2> 709652 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_BF6DB73668EA9BBA-001/shard-1-001
   [junit4]   2> 709653 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 709668 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@327a9510{/,null,AVAILABLE}
   [junit4]   2> 709669 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.e.j.s.ServerConnector Started ServerConnector@6d2fc2e1{HTTP/1.1}{127.0.0.1:60403}
   [junit4]   2> 709669 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.e.j.s.Server Started @712507ms
   [junit4]   2> 709669 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solrconfig=solrconfig.xml, solr.data.dir=hdfs://localhost:58696/hdfs__localhost_58696__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-5.x_solr_build_solr-core_test_J1_temp_solr.cloud.hdfs.HdfsBasicDistributedZk2Test_BF6DB73668EA9BBA-001_tempDir-002_jetty1, hostPort=60403, hostContext=/, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_BF6DB73668EA9BBA-001/shard-1-001/cores}
   [junit4]   2> 709669 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@7b3cb2c6
   [junit4]   2> 709669 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_BF6DB73668EA9BBA-001/shard-1-001'
   [junit4]   2> 709670 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 709670 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 709670 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 709676 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 709676 INFO  (zkCallback-1582-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5c94f01e name:ZooKeeperConnection Watcher:127.0.0.1:34302/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 709676 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 709677 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 709681 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 709681 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_BF6DB73668EA9BBA-001/shard-1-001/solr.xml
   [junit4]   2> 709696 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_BF6DB73668EA9BBA-001/shard-1-001/cores
   [junit4]   2> 709696 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.CoreContainer New CoreContainer 2146791045
   [junit4]   2> 709696 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_BF6DB73668EA9BBA-001/shard-1-001]
   [junit4]   2> 709696 WARN  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_BF6DB73668EA9BBA-001/shard-1-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_BF6DB73668EA9BBA-001/shard-1-001/lib
   [junit4]   2> 709698 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 709700 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 709700 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 709700 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 709700 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:34302/solr
   [junit4]   2> 709700 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 709700 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 709703 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 709707 INFO  (zkCallback-1584-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@4c11ac42 name:ZooKeeperConnection Watcher:127.0.0.1:34302 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 709708 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 709708 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 709711 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x15185de05b70008, likely client has closed socket
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 709716 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [n:127.0.0.1:60403_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 709718 INFO  (zkCallback-1585-thread-1-processing-n:127.0.0.1:60403_) [n:127.0.0.1:60403_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7dceea47 name:ZooKeeperConnection Watcher:127.0.0.1:34302/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 709718 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [n:127.0.0.1:60403_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 709728 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [n:127.0.0.1:60403_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 710741 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [n:127.0.0.1:60403_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:60403_
   [junit4]   2> 710742 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [n:127.0.0.1:60403_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:60403_
   [junit4]   2> 710743 INFO  (zkCallback-1580-thread-1) [    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 710743 INFO  (zkCallback-1585-thread-1-processing-n:127.0.0.1:60403_) [n:127.0.0.1:60403_    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 710743 INFO  (zkCallback-1578-thread-1-processing-n:127.0.0.1:46663_) [n:127.0.0.1:46663_    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 710746 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [n:127.0.0.1:60403_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 710747 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [n:127.0.0.1:60403_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/95004255776604169-127.0.0.1:60403_-n_0000000001
   [junit4]   2> 710749 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [n:127.0.0.1:60403_    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/95004255776604164-127.0.0.1:46663_-n_0000000000 to know if I could be the leader
   [junit4]   2> 710765 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [n:127.0.0.1:60403_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 710765 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [n:127.0.0.1:60403_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 710766 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [n:127.0.0.1:60403_    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_BF6DB73668EA9BBA-001/shard-1-001/cores
   [junit4]   2> 710766 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [n:127.0.0.1:60403_    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {schema=schema.xml, loadOnStartup=true, coreNodeName=, shard=, collection=collection1, name=collection1, config=solrconfig.xml, transient=false, configSetProperties=configsetprops.json, dataDir=data/}
   [junit4]   2> 710767 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [n:127.0.0.1:60403_    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_BF6DB73668EA9BBA-001/shard-1-001/cores/collection1
   [junit4]   2> 710767 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [n:127.0.0.1:60403_    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 710769 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [n:127.0.0.1:60403_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 710769 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[BF6DB73668EA9BBA]) [n:127.0.0.1:60403_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 710771 INFO  (coreLoadExecutor-4056-thread-1-processing-n:127.0.0.1:60403_) [n:127.0.0.1:60403_ c:collection1   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 710771 INFO  (coreLoadExecutor-4056-thread-1-processing-n:127.0.0.1:60403_) [n:127.0.0.1:60403_ c:collection1   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 710773 INFO  (coreLoadExecutor-4056-thread-1-processing-n:127.0.0.1:60403_) [n:127.0.0.1:60403_ c:collection1   x:collection1] o.a.s.c.ZkController Registering watch for collection collection1
   [junit4]   2> 710773 INFO  (OverseerStateUpdate-95004255776604164-127.0.0.1:46663_-n_0000000000) [n:127.0.0.1:46663_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "node_name":"127.0.0.1:60403_",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "base_url":"http://127.0.0.1:60403"} current state version: 4
   [junit4]   2> 710773 INFO  (coreLoadExecutor-4056-thread-1-processing-n:127.0.0.1:60403_) [n:127.0.0.1:60403_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader addZkWatch collection1
   [junit4]   2> 710774 INFO  (OverseerStateUpdate-95004255776604164-127.0.0.1:46663_-n_0000000000) [n:127.0.0.1:46663_    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "node_name":"127.0.0.1:60403_",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "base_url":"http://127.0.0.1:60403"}
   [junit4]   2> 710774 INFO  (OverseerStateUpdate-95004255776604164-127.0.0.1:46663_-n_0000000000) [n:127.0.0.1:46663_    ] o.a.s.c.o.ReplicaMutator Collection already exists with numShards=2
   [junit4]   2> 710774 INFO  (OverseerStateUpdate-95004255776604164-127.0.0.1:46663_-n_0000000000) [n:127.0.0.1:46663_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard2
   [junit4]   2> 710774 INFO  (coreLoadExecutor-4056-thread-1-processing-n:127.0.0.1:60403_) [n:127.0.0.1:60403_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Add data for collection1 ver 0 
   [junit4]   2> 710774 INFO  (coreLoadExecutor-4056-thread-1-processing-n:127.0.0.1:60403_) [n:127.0.0.1:60403_ c:collection1   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 710876 INFO  (OverseerStateUpdate-95004255776604164-127.0.0.1:46663_-n_0000000000) [n:127.0.0.1:46663_    ] o.a.s.c.o.ZkStateWriter going to update_collection /collections/collection1/state.json version: 0
   [junit4]   2> 710877 INFO  (zkCallback-1585-thread-1-processing-n:127.0.0.1:60403_) [n:127.0.0.1:60403_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json for collection collection1 has occurred - updating... (live nodes size: 2)
   [junit4]   2> 710877 INFO  (zkCallback-1585-thread-1-processing-n:127.0.0.1:60403_) [n:127.0.0.1:60403_    ] o.a.s.c.c.ZkStateReader Updating data for collection1 from 0 to 1 
   [junit4]   2> 711774 INFO  (coreLoadExecutor-4056-thread-1-processing-n:127.0.0.1:60403_) [n:127.0.0.1:60403_ c:collection1   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 711774 INFO  (coreLoadExecutor-4056-thread-1-processing-n:127.0.0.1:60403_) [n:127.0.0.1:60403_ c:collection1   x:collection1] o.a.s.c.ZkController Check for collection zkNode:collection1
   [junit4]   2> 711775 INFO  (coreLoadExecutor-4056-thread-1-processing-n:127.0.0.1:60403_) [n:127.0.0.1:60403_ c:collection1   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 711775 INFO  (coreLoadExecutor-4056-thread-1-processing-n:127.0.0.1:60403_) [n:127.0.0.1:60403_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/collection1
   [junit4]   2> 711776 INFO  (coreLoadExecutor-4056-thread-1-processing-n:127.0.0.1:60403_) [n:127.0.0.1:60403_ c:collection1   x:coll

[...truncated too long message...]

terceptor.java:1813)
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.exclusiveUnregisterMBean(DefaultMBeanServerInterceptor.java:430)
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.unregisterMBean(DefaultMBeanServerInterceptor.java:415)
   [junit4]   2> 	at com.sun.jmx.mbeanserver.JmxMBeanServer.unregisterMBean(JmxMBeanServer.java:546)
   [junit4]   2> 	at org.apache.hadoop.metrics2.util.MBeans.unregister(MBeans.java:81)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.stopMBeans(MetricsSourceAdapter.java:227)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.stop(MetricsSourceAdapter.java:212)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.stopSources(MetricsSystemImpl.java:461)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.stop(MetricsSystemImpl.java:212)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.shutdown(MetricsSystemImpl.java:592)
   [junit4]   2> 	at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.shutdownInstance(DefaultMetricsSystem.java:72)
   [junit4]   2> 	at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.shutdown(DefaultMetricsSystem.java:68)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.metrics.NameNodeMetrics.shutdown(NameNodeMetrics.java:145)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.NameNode.stop(NameNode.java:822)
   [junit4]   2> 	at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1720)
   [junit4]   2> 	at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1699)
   [junit4]   2> 	at org.apache.solr.cloud.hdfs.HdfsTestUtil.teardownClass(HdfsTestUtil.java:198)
   [junit4]   2> 	at org.apache.solr.cloud.hdfs.HdfsBasicDistributedZk2Test.teardownClass(HdfsBasicDistributedZk2Test.java:47)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   [junit4]   2> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2> 	at java.lang.reflect.Method.invoke(Method.java:606)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:834)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> Caused by: java.lang.NullPointerException
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.blockmanagement.BlocksMap.size(BlocksMap.java:198)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.getTotalBlocks(BlockManager.java:3291)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlocksTotal(FSNamesystem.java:6223)
   [junit4]   2> 	... 54 more
   [junit4]   2> 809787 INFO  (SUITE-HdfsBasicDistributedZk2Test-seed#[BF6DB73668EA9BBA]-worker) [n:127.0.0.1:59573_ c:collection1 s:shard1 r:core_node4 x:collection1] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_BF6DB73668EA9BBA-001
   [junit4]   2> Dec 09, 2015 8:33:22 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 3 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene54): {foo_b=FST50, n_l1=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), intDefault=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), n_d1=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), n_tdt1=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), range_facet_i_dv=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), n_ti1=PostingsFormat(name=Memory doPackFST= false), _version_=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), multiDefault=Lucene50(blocksize=128), id=PostingsFormat(name=Memory doPackFST= false), foo_f=FST50, n_tl1=Lucene50(blocksize=128), rnd_b=PostingsFormat(name=Memory doPackFST= false), range_facet_l=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), foo_d=PostingsFormat(name=Memory doPackFST= false), n_td1=Lucene50(blocksize=128), oddField_s=FST50, timestamp=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), text=FST50, SubjectTerms_mfacet=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), n_tf1=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), n_f1=Lucene50(blocksize=128), n_dt1=Lucene50(blocksize=128), range_facet_l_dv=PostingsFormat(name=Memory doPackFST= false), other_tl1=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), a_t=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), a_i1=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128)))}, docValues:{range_facet_l_dv=DocValuesFormat(name=Lucene54), timestamp=DocValuesFormat(name=Memory), range_facet_i_dv=DocValuesFormat(name=Memory)}, sim=RandomSimilarityProvider(queryNorm=true,coord=crazy): {}, locale=it_CH, timezone=Asia/Riyadh
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_80 (64-bit)/cpus=4,threads=5,free=268238792,total=466616320
   [junit4]   2> NOTE: All tests run in this JVM: [HighlighterMaxOffsetTest, TestSolrCoreProperties, TestSortByMinMaxFunction, SpellingQueryConverterTest, TestSurroundQueryParser, AnalyticsMergeStrategyTest, CollectionsAPIDistributedZkTest, TestStressVersions, SimplePostToolTest, ChaosMonkeySafeLeaderTest, TestJsonFacets, DistanceFunctionTest, TestRandomRequestDistribution, TestCloudPivotFacet, MoreLikeThisHandlerTest, TestRemoveLastDynamicCopyField, UpdateRequestProcessorFactoryTest, TestWordDelimiterFilterFactory, TestSolrConfigHandlerCloud, TestLRUStatsCache, TestSolrQueryParserDefaultOperatorResource, TestRawTransformer, TestHashQParserPlugin, HdfsThreadLeakTest, TestTrie, TestReplicationHandlerBackup, TestDistributedGrouping, TestLeaderElectionZkExpiry, TestDFRSimilarityFactory, BasicFunctionalityTest, TestMissingGroups, IndexBasedSpellCheckerTest, HdfsBasicDistributedZk2Test]
   [junit4] Completed [102/553 (1!)] on J1 in 115.89s, 1 test, 1 error <<< FAILURES!

[...truncated 1444 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:801: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:738: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1456: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1010: There were test failures: 553 suites (7 ignored), 2188 tests, 1 error, 33 ignored (18 assumptions) [seed: BF6DB73668EA9BBA]

Total time: 203 minutes 52 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

Re: [JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1040 - Still Failing

[Mikhail Khludnev <mk...@griddynamics.com>]

Got. I just didn't have jdk7 after last laptop crash. 5x build on jdk8
caught syntax errors, that's why I relaxed after fixing compile errors and
run tests. I didn't catch rt.jar difference. Now I'm able to test on jdk7.
Awaiting for CI.

On Tue, Dec 8, 2015 at 5:25 PM, david.w.smiley@gmail.com <
david.w.smiley@gmail.com> wrote:

> Mikhail,
> In the future, particularly when committing code that uses Java 8
> features, be sure to compile & run tests even on the 5x branch after
> back-porting but before committing.  From time to time this fails for
> stuff, like easy-to-forget cases of not using ‘final’.
> ~ David
>
> On Tue, Dec 8, 2015 at 2:21 AM Mikhail Khludnev <
> mkhludnev@griddynamics.com> wrote:
>
>> It's mine. Looking into.
>>
>> On Tue, Dec 8, 2015 at 9:35 AM, Apache Jenkins Server <
>> jenkins@builds.apache.org> wrote:
>>
>>> Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1040/
>>>
>>> All tests passed
>>>
>>> Build Log:
>>> [...truncated 7450 lines...]
>>>     [javac] Compiling 6 source files to
>>> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/join/classes/test
>>>     [javac]
>>> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/join/src/test/org/apache/lucene/search/join/TestJoinUtil.java:954:
>>> error: cannot find symbol
>>>     [javac]         assert nextInt ==
>>> Integer.parseUnsignedInt(uniqueRandomValue,16);
>>>     [javac]                                  ^
>>>     [javac]   symbol:   method parseUnsignedInt(String,int)
>>>     [javac]   location: class Integer
>>>     [javac]
>>> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/join/src/test/org/apache/lucene/search/join/TestJoinUtil.java:1206:
>>> error: cannot find symbol
>>>     [javac]     final int linkInt =
>>> Integer.parseUnsignedInt(linkValue,16);
>>>     [javac]                                ^
>>>     [javac]   symbol:   method parseUnsignedInt(String,int)
>>>     [javac]   location: class Integer
>>>     [javac] Note:
>>> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/join/src/test/org/apache/lucene/search/join/TestBlockJoin.java
>>> uses or overrides a deprecated API.
>>>     [javac] Note: Recompile with -Xlint:deprecation for details.
>>>     [javac] 2 errors
>>>
>>> BUILD FAILED
>>> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:801:
>>> The following error occurred while executing this line:
>>> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:738:
>>> The following error occurred while executing this line:
>>> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59:
>>> The following error occurred while executing this line:
>>> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build.xml:472:
>>> The following error occurred while executing this line:
>>> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:2252:
>>> The following error occurred while executing this line:
>>> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/module-build.xml:58:
>>> The following error occurred while executing this line:
>>> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/module-build.xml:55:
>>> The following error occurred while executing this line:
>>> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:818:
>>> The following error occurred while executing this line:
>>> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:832:
>>> The following error occurred while executing this line:
>>> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1968:
>>> Compile failed; see the compiler error output for details.
>>>
>>> Total time: 106 minutes 8 seconds
>>> Build step 'Invoke Ant' marked build as failure
>>> Archiving artifacts
>>> Recording test results
>>> Email was triggered for: Failure - Any
>>> Sending email for trigger: Failure - Any
>>>
>>>
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>> For additional commands, e-mail: dev-help@lucene.apache.org
>>>
>>
>>
>>
>> --
>> Sincerely yours
>> Mikhail Khludnev
>> Principal Engineer,
>> Grid Dynamics
>>
>> <http://www.griddynamics.com>
>> <mk...@griddynamics.com>
>>
> --
> Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
> LinkedIn: http://linkedin.com/in/davidwsmiley | Book:
> http://www.solrenterprisesearchserver.com
>



-- 
Sincerely yours
Mikhail Khludnev
Principal Engineer,
Grid Dynamics

<http://www.griddynamics.com>
<mk...@griddynamics.com>

Re: [JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1040 - Still Failing

["david.w.smiley@gmail.com" <da...@gmail.com>]

Mikhail,
In the future, particularly when committing code that uses Java 8 features,
be sure to compile & run tests even on the 5x branch after back-porting but
before committing.  From time to time this fails for stuff, like
easy-to-forget cases of not using ‘final’.
~ David

On Tue, Dec 8, 2015 at 2:21 AM Mikhail Khludnev <mk...@griddynamics.com>
wrote:

> It's mine. Looking into.
>
> On Tue, Dec 8, 2015 at 9:35 AM, Apache Jenkins Server <
> jenkins@builds.apache.org> wrote:
>
>> Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1040/
>>
>> All tests passed
>>
>> Build Log:
>> [...truncated 7450 lines...]
>>     [javac] Compiling 6 source files to
>> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/join/classes/test
>>     [javac]
>> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/join/src/test/org/apache/lucene/search/join/TestJoinUtil.java:954:
>> error: cannot find symbol
>>     [javac]         assert nextInt ==
>> Integer.parseUnsignedInt(uniqueRandomValue,16);
>>     [javac]                                  ^
>>     [javac]   symbol:   method parseUnsignedInt(String,int)
>>     [javac]   location: class Integer
>>     [javac]
>> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/join/src/test/org/apache/lucene/search/join/TestJoinUtil.java:1206:
>> error: cannot find symbol
>>     [javac]     final int linkInt =
>> Integer.parseUnsignedInt(linkValue,16);
>>     [javac]                                ^
>>     [javac]   symbol:   method parseUnsignedInt(String,int)
>>     [javac]   location: class Integer
>>     [javac] Note:
>> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/join/src/test/org/apache/lucene/search/join/TestBlockJoin.java
>> uses or overrides a deprecated API.
>>     [javac] Note: Recompile with -Xlint:deprecation for details.
>>     [javac] 2 errors
>>
>> BUILD FAILED
>> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:801:
>> The following error occurred while executing this line:
>> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:738:
>> The following error occurred while executing this line:
>> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59:
>> The following error occurred while executing this line:
>> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build.xml:472:
>> The following error occurred while executing this line:
>> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:2252:
>> The following error occurred while executing this line:
>> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/module-build.xml:58:
>> The following error occurred while executing this line:
>> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/module-build.xml:55:
>> The following error occurred while executing this line:
>> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:818:
>> The following error occurred while executing this line:
>> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:832:
>> The following error occurred while executing this line:
>> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1968:
>> Compile failed; see the compiler error output for details.
>>
>> Total time: 106 minutes 8 seconds
>> Build step 'Invoke Ant' marked build as failure
>> Archiving artifacts
>> Recording test results
>> Email was triggered for: Failure - Any
>> Sending email for trigger: Failure - Any
>>
>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: dev-help@lucene.apache.org
>>
>
>
>
> --
> Sincerely yours
> Mikhail Khludnev
> Principal Engineer,
> Grid Dynamics
>
> <http://www.griddynamics.com>
> <mk...@griddynamics.com>
>
-- 
Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
LinkedIn: http://linkedin.com/in/davidwsmiley | Book:
http://www.solrenterprisesearchserver.com

Re: [JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1040 - Still Failing

[Mikhail Khludnev <mk...@griddynamics.com>]

It's mine. Looking into.

On Tue, Dec 8, 2015 at 9:35 AM, Apache Jenkins Server <
jenkins@builds.apache.org> wrote:

> Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1040/
>
> All tests passed
>
> Build Log:
> [...truncated 7450 lines...]
>     [javac] Compiling 6 source files to
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/join/classes/test
>     [javac]
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/join/src/test/org/apache/lucene/search/join/TestJoinUtil.java:954:
> error: cannot find symbol
>     [javac]         assert nextInt ==
> Integer.parseUnsignedInt(uniqueRandomValue,16);
>     [javac]                                  ^
>     [javac]   symbol:   method parseUnsignedInt(String,int)
>     [javac]   location: class Integer
>     [javac]
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/join/src/test/org/apache/lucene/search/join/TestJoinUtil.java:1206:
> error: cannot find symbol
>     [javac]     final int linkInt = Integer.parseUnsignedInt(linkValue,16);
>     [javac]                                ^
>     [javac]   symbol:   method parseUnsignedInt(String,int)
>     [javac]   location: class Integer
>     [javac] Note:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/join/src/test/org/apache/lucene/search/join/TestBlockJoin.java
> uses or overrides a deprecated API.
>     [javac] Note: Recompile with -Xlint:deprecation for details.
>     [javac] 2 errors
>
> BUILD FAILED
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:801:
> The following error occurred while executing this line:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:738:
> The following error occurred while executing this line:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59:
> The following error occurred while executing this line:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build.xml:472:
> The following error occurred while executing this line:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:2252:
> The following error occurred while executing this line:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/module-build.xml:58:
> The following error occurred while executing this line:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/module-build.xml:55:
> The following error occurred while executing this line:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:818:
> The following error occurred while executing this line:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:832:
> The following error occurred while executing this line:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1968:
> Compile failed; see the compiler error output for details.
>
> Total time: 106 minutes 8 seconds
> Build step 'Invoke Ant' marked build as failure
> Archiving artifacts
> Recording test results
> Email was triggered for: Failure - Any
> Sending email for trigger: Failure - Any
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>



-- 
Sincerely yours
Mikhail Khludnev
Principal Engineer,
Grid Dynamics

<http://www.griddynamics.com>
<mk...@griddynamics.com>

[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1040 - Still Failing

[Apache Jenkins Server <je...@builds.apache.org>]

Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1040/

All tests passed

Build Log:
[...truncated 7450 lines...]
    [javac] Compiling 6 source files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/join/classes/test
    [javac] /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/join/src/test/org/apache/lucene/search/join/TestJoinUtil.java:954: error: cannot find symbol
    [javac]         assert nextInt == Integer.parseUnsignedInt(uniqueRandomValue,16);
    [javac]                                  ^
    [javac]   symbol:   method parseUnsignedInt(String,int)
    [javac]   location: class Integer
    [javac] /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/join/src/test/org/apache/lucene/search/join/TestJoinUtil.java:1206: error: cannot find symbol
    [javac]     final int linkInt = Integer.parseUnsignedInt(linkValue,16);
    [javac]                                ^
    [javac]   symbol:   method parseUnsignedInt(String,int)
    [javac]   location: class Integer
    [javac] Note: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/join/src/test/org/apache/lucene/search/join/TestBlockJoin.java uses or overrides a deprecated API.
    [javac] Note: Recompile with -Xlint:deprecation for details.
    [javac] 2 errors

BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:801: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:738: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build.xml:472: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:2252: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/module-build.xml:58: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/module-build.xml:55: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:818: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:832: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1968: Compile failed; see the compiler error output for details.

Total time: 106 minutes 8 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1039 - Still Failing

[Apache Jenkins Server <je...@builds.apache.org>]

Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1039/

3 tests failed.
FAILED:  org.apache.solr.cloud.TestAuthenticationFramework.testCollectionCreateSearchDelete

Error Message:
Address already in use

Stack Trace:
java.net.BindException: Address already in use
	at __randomizedtesting.SeedInfo.seed([6B4A28DD8930C974:C8B086780ED823D1]:0)
	at sun.nio.ch.Net.bind0(Native Method)
	at sun.nio.ch.Net.bind(Net.java:463)
	at sun.nio.ch.Net.bind(Net.java:455)
	at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223)
	at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
	at org.eclipse.jetty.server.ServerConnector.open(ServerConnector.java:321)
	at org.eclipse.jetty.server.AbstractNetworkConnector.doStart(AbstractNetworkConnector.java:80)
	at org.eclipse.jetty.server.ServerConnector.doStart(ServerConnector.java:236)
	at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
	at org.eclipse.jetty.server.Server.doStart(Server.java:366)
	at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
	at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:407)
	at org.apache.solr.cloud.MiniSolrCloudCluster.startJettySolrRunner(MiniSolrCloudCluster.java:357)
	at org.apache.solr.cloud.TestMiniSolrCloudCluster.testCollectionCreateSearchDelete(TestMiniSolrCloudCluster.java:190)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.TestAuthenticationFramework

Error Message:
10 threads leaked from SUITE scope at org.apache.solr.cloud.TestAuthenticationFramework:     1) Thread[id=50657, name=qtp1598845069-50657, state=TIMED_WAITING, group=TGRP-TestAuthenticationFramework]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)         at java.lang.Thread.run(Thread.java:745)    2) Thread[id=50661, name=Scheduler-304156586, state=TIMED_WAITING, group=TGRP-TestAuthenticationFramework]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1090)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)    3) Thread[id=50656, name=qtp1598845069-50656, state=TIMED_WAITING, group=TGRP-TestAuthenticationFramework]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)         at java.lang.Thread.run(Thread.java:745)    4) Thread[id=50655, name=qtp1598845069-50655, state=TIMED_WAITING, group=TGRP-TestAuthenticationFramework]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)         at java.lang.Thread.run(Thread.java:745)    5) Thread[id=50658, name=qtp1598845069-50658, state=TIMED_WAITING, group=TGRP-TestAuthenticationFramework]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)         at java.lang.Thread.run(Thread.java:745)    6) Thread[id=50654, name=qtp1598845069-50654, state=TIMED_WAITING, group=TGRP-TestAuthenticationFramework]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)         at java.lang.Thread.run(Thread.java:745)    7) Thread[id=50660, name=qtp1598845069-50660, state=TIMED_WAITING, group=TGRP-TestAuthenticationFramework]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)         at java.lang.Thread.run(Thread.java:745)    8) Thread[id=50653, name=qtp1598845069-50653, state=TIMED_WAITING, group=TGRP-TestAuthenticationFramework]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)         at java.lang.Thread.run(Thread.java:745)    9) Thread[id=50659, name=qtp1598845069-50659, state=TIMED_WAITING, group=TGRP-TestAuthenticationFramework]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)         at java.lang.Thread.run(Thread.java:745)   10) Thread[id=50662, name=org.eclipse.jetty.server.session.HashSessionManager@25ab2976Timer, state=TIMED_WAITING, group=TGRP-TestAuthenticationFramework]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1090)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 10 threads leaked from SUITE scope at org.apache.solr.cloud.TestAuthenticationFramework: 
   1) Thread[id=50657, name=qtp1598845069-50657, state=TIMED_WAITING, group=TGRP-TestAuthenticationFramework]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)
        at java.lang.Thread.run(Thread.java:745)
   2) Thread[id=50661, name=Scheduler-304156586, state=TIMED_WAITING, group=TGRP-TestAuthenticationFramework]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1090)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
   3) Thread[id=50656, name=qtp1598845069-50656, state=TIMED_WAITING, group=TGRP-TestAuthenticationFramework]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)
        at java.lang.Thread.run(Thread.java:745)
   4) Thread[id=50655, name=qtp1598845069-50655, state=TIMED_WAITING, group=TGRP-TestAuthenticationFramework]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)
        at java.lang.Thread.run(Thread.java:745)
   5) Thread[id=50658, name=qtp1598845069-50658, state=TIMED_WAITING, group=TGRP-TestAuthenticationFramework]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)
        at java.lang.Thread.run(Thread.java:745)
   6) Thread[id=50654, name=qtp1598845069-50654, state=TIMED_WAITING, group=TGRP-TestAuthenticationFramework]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)
        at java.lang.Thread.run(Thread.java:745)
   7) Thread[id=50660, name=qtp1598845069-50660, state=TIMED_WAITING, group=TGRP-TestAuthenticationFramework]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)
        at java.lang.Thread.run(Thread.java:745)
   8) Thread[id=50653, name=qtp1598845069-50653, state=TIMED_WAITING, group=TGRP-TestAuthenticationFramework]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)
        at java.lang.Thread.run(Thread.java:745)
   9) Thread[id=50659, name=qtp1598845069-50659, state=TIMED_WAITING, group=TGRP-TestAuthenticationFramework]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)
        at java.lang.Thread.run(Thread.java:745)
  10) Thread[id=50662, name=org.eclipse.jetty.server.session.HashSessionManager@25ab2976Timer, state=TIMED_WAITING, group=TGRP-TestAuthenticationFramework]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1090)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
	at __randomizedtesting.SeedInfo.seed([6B4A28DD8930C974]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.TestAuthenticationFramework

Error Message:
There are still zombie threads that couldn't be terminated:    1) Thread[id=50661, name=Scheduler-304156586, state=TIMED_WAITING, group=TGRP-TestAuthenticationFramework]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1090)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)    2) Thread[id=50662, name=org.eclipse.jetty.server.session.HashSessionManager@25ab2976Timer, state=TIMED_WAITING, group=TGRP-TestAuthenticationFramework]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1090)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   1) Thread[id=50661, name=Scheduler-304156586, state=TIMED_WAITING, group=TGRP-TestAuthenticationFramework]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1090)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
   2) Thread[id=50662, name=org.eclipse.jetty.server.session.HashSessionManager@25ab2976Timer, state=TIMED_WAITING, group=TGRP-TestAuthenticationFramework]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1090)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
	at __randomizedtesting.SeedInfo.seed([6B4A28DD8930C974]:0)




Build Log:
[...truncated 10595 lines...]
   [junit4] Suite: org.apache.solr.cloud.TestAuthenticationFramework
   [junit4]   2> 855113 INFO  (TEST-TestAuthenticationFramework.testErrorsInShutdown-seed#[6B4A28DD8930C974]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 855114 INFO  (Thread-48265) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 855114 INFO  (Thread-48265) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 855214 INFO  (TEST-TestAuthenticationFramework.testErrorsInShutdown-seed#[6B4A28DD8930C974]) [    ] o.a.s.c.ZkTestServer start zk server on port:51098
   [junit4]   2> 855214 INFO  (TEST-TestAuthenticationFramework.testErrorsInShutdown-seed#[6B4A28DD8930C974]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 855214 INFO  (TEST-TestAuthenticationFramework.testErrorsInShutdown-seed#[6B4A28DD8930C974]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 855223 INFO  (zkCallback-386-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7c315d15 name:ZooKeeperConnection Watcher:127.0.0.1:51098 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 855223 INFO  (TEST-TestAuthenticationFramework.testErrorsInShutdown-seed#[6B4A28DD8930C974]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 855223 INFO  (TEST-TestAuthenticationFramework.testErrorsInShutdown-seed#[6B4A28DD8930C974]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 855224 INFO  (TEST-TestAuthenticationFramework.testErrorsInShutdown-seed#[6B4A28DD8930C974]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr/solr.xml
   [junit4]   2> 855230 INFO  (jetty-launcher-385-thread-1) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 855230 INFO  (jetty-launcher-385-thread-2) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 855231 INFO  (jetty-launcher-385-thread-3) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 855234 INFO  (jetty-launcher-385-thread-3) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@689a6a7e{/solr,null,AVAILABLE}
   [junit4]   2> 855234 INFO  (jetty-launcher-385-thread-1) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@37db9ed2{/solr,null,AVAILABLE}
   [junit4]   2> 855234 INFO  (jetty-launcher-385-thread-2) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@339809a4{/solr,null,AVAILABLE}
   [junit4]   2> 855238 INFO  (jetty-launcher-385-thread-1) [    ] o.e.j.s.ServerConnector Started ServerConnector@6d926c48{HTTP/1.1}{127.0.0.1:55066}
   [junit4]   2> 855238 INFO  (jetty-launcher-385-thread-1) [    ] o.e.j.s.Server Started @858130ms
   [junit4]   2> 855238 INFO  (jetty-launcher-385-thread-1) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostPort=55066, hostContext=/solr}
   [junit4]   2> 855238 INFO  (jetty-launcher-385-thread-1) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@3827de24
   [junit4]   2> 855238 INFO  (jetty-launcher-385-thread-1) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestAuthenticationFramework_6B4A28DD8930C974-001/tempDir-001/node1'
   [junit4]   2> 855238 INFO  (jetty-launcher-385-thread-1) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 855238 INFO  (jetty-launcher-385-thread-1) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 855239 INFO  (jetty-launcher-385-thread-1) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 855239 INFO  (jetty-launcher-385-thread-1) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 855239 INFO  (jetty-launcher-385-thread-2) [    ] o.e.j.s.ServerConnector Started ServerConnector@65f9f092{HTTP/1.1}{127.0.0.1:48544}
   [junit4]   2> 855240 INFO  (jetty-launcher-385-thread-2) [    ] o.e.j.s.Server Started @858132ms
   [junit4]   2> 855240 INFO  (jetty-launcher-385-thread-2) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostPort=48544, hostContext=/solr}
   [junit4]   2> 855240 INFO  (jetty-launcher-385-thread-3) [    ] o.e.j.s.ServerConnector Started ServerConnector@c999844{HTTP/1.1}{127.0.0.1:55562}
   [junit4]   2> 855240 INFO  (jetty-launcher-385-thread-3) [    ] o.e.j.s.Server Started @858132ms
   [junit4]   2> 855240 INFO  (jetty-launcher-385-thread-3) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostPort=55562, hostContext=/solr}
   [junit4]   2> 855240 INFO  (jetty-launcher-385-thread-2) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@3827de24
   [junit4]   2> 855240 INFO  (jetty-launcher-385-thread-2) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestAuthenticationFramework_6B4A28DD8930C974-001/tempDir-001/node2'
   [junit4]   2> 855241 INFO  (jetty-launcher-385-thread-2) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 855241 INFO  (jetty-launcher-385-thread-3) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@3827de24
   [junit4]   2> 855241 INFO  (jetty-launcher-385-thread-2) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 855241 INFO  (jetty-launcher-385-thread-3) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestAuthenticationFramework_6B4A28DD8930C974-001/tempDir-001/node3'
   [junit4]   2> 855241 INFO  (jetty-launcher-385-thread-3) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 855241 INFO  (jetty-launcher-385-thread-2) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 855241 INFO  (jetty-launcher-385-thread-3) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 855241 INFO  (jetty-launcher-385-thread-3) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 855241 INFO  (jetty-launcher-385-thread-2) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 855242 INFO  (jetty-launcher-385-thread-3) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 855244 INFO  (zkCallback-387-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7f2809d6 name:ZooKeeperConnection Watcher:127.0.0.1:51098/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 855244 INFO  (jetty-launcher-385-thread-1) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 855244 INFO  (zkCallback-388-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@579a9cba name:ZooKeeperConnection Watcher:127.0.0.1:51098/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 855244 INFO  (jetty-launcher-385-thread-2) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 855244 INFO  (jetty-launcher-385-thread-1) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 855244 INFO  (jetty-launcher-385-thread-2) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 855245 INFO  (jetty-launcher-385-thread-1) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 855250 INFO  (jetty-launcher-385-thread-2) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 855250 INFO  (zkCallback-389-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@44c4ad4c name:ZooKeeperConnection Watcher:127.0.0.1:51098/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 855250 INFO  (jetty-launcher-385-thread-3) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 855250 INFO  (jetty-launcher-385-thread-3) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 855252 INFO  (jetty-launcher-385-thread-3) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 855267 INFO  (jetty-launcher-385-thread-2) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestAuthenticationFramework_6B4A28DD8930C974-001/tempDir-001/node2/.
   [junit4]   2> 855267 INFO  (jetty-launcher-385-thread-2) [    ] o.a.s.c.CoreContainer New CoreContainer 399278526
   [junit4]   2> 855267 INFO  (jetty-launcher-385-thread-2) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestAuthenticationFramework_6B4A28DD8930C974-001/tempDir-001/node2]
   [junit4]   2> 855267 WARN  (jetty-launcher-385-thread-2) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestAuthenticationFramework_6B4A28DD8930C974-001/tempDir-001/node2/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestAuthenticationFramework_6B4A28DD8930C974-001/tempDir-001/node2/lib
   [junit4]   2> 855268 INFO  (jetty-launcher-385-thread-2) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 855268 INFO  (jetty-launcher-385-thread-1) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestAuthenticationFramework_6B4A28DD8930C974-001/tempDir-001/node1/.
   [junit4]   2> 855268 INFO  (jetty-launcher-385-thread-1) [    ] o.a.s.c.CoreContainer New CoreContainer 547657681
   [junit4]   2> 855269 INFO  (jetty-launcher-385-thread-1) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestAuthenticationFramework_6B4A28DD8930C974-001/tempDir-001/node1]
   [junit4]   2> 855269 WARN  (jetty-launcher-385-thread-1) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestAuthenticationFramework_6B4A28DD8930C974-001/tempDir-001/node1/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestAuthenticationFramework_6B4A28DD8930C974-001/tempDir-001/node1/lib
   [junit4]   2> 855269 INFO  (jetty-launcher-385-thread-1) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 855270 INFO  (jetty-launcher-385-thread-1) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 855270 INFO  (jetty-launcher-385-thread-2) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 855270 INFO  (jetty-launcher-385-thread-1) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 855270 INFO  (jetty-launcher-385-thread-1) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 855270 INFO  (jetty-launcher-385-thread-2) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 855270 INFO  (jetty-launcher-385-thread-2) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 855271 INFO  (jetty-launcher-385-thread-2) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:51098/solr
   [junit4]   2> 855271 INFO  (jetty-launcher-385-thread-1) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:51098/solr
   [junit4]   2> 855271 INFO  (jetty-launcher-385-thread-2) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 855272 INFO  (jetty-launcher-385-thread-1) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 855272 INFO  (jetty-launcher-385-thread-2) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 855272 INFO  (jetty-launcher-385-thread-1) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 855273 INFO  (jetty-launcher-385-thread-1) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 855273 INFO  (jetty-launcher-385-thread-3) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestAuthenticationFramework_6B4A28DD8930C974-001/tempDir-001/node3/.
   [junit4]   2> 855273 INFO  (jetty-launcher-385-thread-3) [    ] o.a.s.c.CoreContainer New CoreContainer 54608992
   [junit4]   2> 855273 INFO  (jetty-launcher-385-thread-3) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestAuthenticationFramework_6B4A28DD8930C974-001/tempDir-001/node3]
   [junit4]   2> 855273 INFO  (jetty-launcher-385-thread-2) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 855273 WARN  (jetty-launcher-385-thread-3) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestAuthenticationFramework_6B4A28DD8930C974-001/tempDir-001/node3/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestAuthenticationFramework_6B4A28DD8930C974-001/tempDir-001/node3/lib
   [junit4]   2> 855274 INFO  (jetty-launcher-385-thread-3) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 855274 INFO  (zkCallback-393-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@40e303e4 name:ZooKeeperConnection Watcher:127.0.0.1:51098 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 855274 INFO  (jetty-launcher-385-thread-3) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 855274 INFO  (jetty-launcher-385-thread-1) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 855275 INFO  (jetty-launcher-385-thread-1) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 855275 INFO  (jetty-launcher-385-thread-3) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 855275 INFO  (jetty-launcher-385-thread-3) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 855275 INFO  (jetty-launcher-385-thread-3) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:51098/solr
   [junit4]   2> 855275 INFO  (jetty-launcher-385-thread-3) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 855275 INFO  (jetty-launcher-385-thread-3) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 855275 INFO  (zkCallback-392-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1ea0ae38 name:ZooKeeperConnection Watcher:127.0.0.1:51098 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 855280 INFO  (jetty-launcher-385-thread-2) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 855280 INFO  (jetty-launcher-385-thread-3) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 855280 INFO  (jetty-launcher-385-thread-2) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 855281 INFO  (jetty-launcher-385-thread-1) [n:127.0.0.1:55066_solr    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 855282 INFO  (zkCallback-395-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@43833910 name:ZooKeeperConnection Watcher:127.0.0.1:51098 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 855282 INFO  (jetty-launcher-385-thread-3) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 855283 INFO  (jetty-launcher-385-thread-3) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 855291 INFO  (jetty-launcher-385-thread-2) [n:127.0.0.1:48544_solr    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 855291 INFO  (zkCallback-396-thread-1-processing-n:127.0.0.1:55066_solr) [n:127.0.0.1:55066_solr    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@703a309f name:ZooKeeperConnection Watcher:127.0.0.1:51098/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 855291 INFO  (jetty-launcher-385-thread-1) [n:127.0.0.1:55066_solr    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 855291 INFO  (zkCallback-397-thread-1-processing-n:127.0.0.1:48544_solr) [n:127.0.0.1:48544_solr    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7073b9fc name:ZooKeeperConnection Watcher:127.0.0.1:51098/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 855291 INFO  (jetty-launcher-385-thread-2) [n:127.0.0.1:48544_solr    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 855292 INFO  (jetty-launcher-385-thread-3) [n:127.0.0.1:55562_solr    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 855294 INFO  (jetty-launcher-385-thread-1) [n:127.0.0.1:55066_solr    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 855294 INFO  (jetty-launcher-385-thread-2) [n:127.0.0.1:48544_solr    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 855294 INFO  (zkCallback-398-thread-1-processing-n:127.0.0.1:55562_solr) [n:127.0.0.1:55562_solr    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@685b0835 name:ZooKeeperConnection Watcher:127.0.0.1:51098/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 855294 INFO  (jetty-launcher-385-thread-3) [n:127.0.0.1:55562_solr    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 855300 INFO  (jetty-launcher-385-thread-1) [n:127.0.0.1:55066_solr    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 855300 INFO  (jetty-launcher-385-thread-2) [n:127.0.0.1:48544_solr    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 855301 INFO  (jetty-launcher-385-thread-3) [n:127.0.0.1:55562_solr    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 855307 INFO  (jetty-launcher-385-thread-1) [n:127.0.0.1:55066_solr    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 855308 INFO  (jetty-launcher-385-thread-2) [n:127.0.0.1:48544_solr    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 855308 INFO  (jetty-launcher-385-thread-3) [n:127.0.0.1:55562_solr    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 855311 INFO  (jetty-launcher-385-thread-2) [n:127.0.0.1:48544_solr    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 855311 INFO  (jetty-launcher-385-thread-1) [n:127.0.0.1:55066_solr    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 855316 INFO  (jetty-launcher-385-thread-3) [n:127.0.0.1:55562_solr    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 855316 INFO  (jetty-launcher-385-thread-2) [n:127.0.0.1:48544_solr    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 855320 INFO  (jetty-launcher-385-thread-1) [n:127.0.0.1:55066_solr    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 855320 INFO  (jetty-launcher-385-thread-3) [n:127.0.0.1:55562_solr    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 855320 INFO  (jetty-launcher-385-thread-2) [n:127.0.0.1:48544_solr    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 855325 INFO  (jetty-launcher-385-thread-3) [n:127.0.0.1:55562_solr    ] o.a.s.c.c.SolrZkClient makePath: /collections
   [junit4]   2> 855325 INFO  (jetty-launcher-385-thread-2) [n:127.0.0.1:48544_solr    ] o.a.s.c.c.SolrZkClient makePath: /collections
   [junit4]   2> 855325 INFO  (jetty-launcher-385-thread-1) [n:127.0.0.1:55066_solr    ] o.a.s.c.c.SolrZkClient makePath: /collections
   [junit4]   2> 855331 INFO  (jetty-launcher-385-thread-2) [n:127.0.0.1:48544_solr    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 855331 INFO  (jetty-launcher-385-thread-3) [n:127.0.0.1:55562_solr    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 855332 INFO  (jetty-launcher-385-thread-1) [n:127.0.0.1:55066_solr    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 855334 INFO  (jetty-launcher-385-thread-2) [n:127.0.0.1:48544_solr    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 855335 INFO  (jetty-launcher-385-thread-3) [n:127.0.0.1:55562_solr    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 855335 INFO  (jetty-launcher-385-thread-1) [n:127.0.0.1:55066_solr    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 855337 INFO  (jetty-launcher-385-thread-3) [n:127.0.0.1:55562_solr    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 855337 INFO  (jetty-launcher-385-thread-2) [n:127.0.0.1:48544_solr    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 855337 INFO  (jetty-launcher-385-thread-1) [n:127.0.0.1:55066_solr    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 855339 INFO  (jetty-launcher-385-thread-2) [n:127.0.0.1:48544_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:48544_solr
   [junit4]   2> 855339 INFO  (jetty-launcher-385-thread-3) [n:127.0.0.1:55562_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:55562_solr
   [junit4]   2> 855340 INFO  (jetty-launcher-385-thread-1) [n:127.0.0.1:55066_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:55066_solr
   [junit4]   2> 855340 INFO  (jetty-launcher-385-thread-2) [n:127.0.0.1:48544_solr    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:48544_solr
   [junit4]   2> 855341 INFO  (jetty-launcher-385-thread-3) [n:127.0.0.1:55562_solr    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:55562_solr
   [junit4]   2> 855341 INFO  (jetty-launcher-385-thread-1) [n:127.0.0.1:55066_solr    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:55066_solr
   [junit4]   2> 855343 INFO  (jetty-launcher-385-thread-2) [n:127.0.0.1:48544_solr    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 855344 INFO  (jetty-launcher-385-thread-3) [n:127.0.0.1:55562_solr    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 855346 INFO  (jetty-launcher-385-thread-2) [n:127.0.0.1:48544_solr    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 855346 INFO  (jetty-launcher-385-thread-3) [n:127.0.0.1:55562_solr    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 855346 INFO  (jetty-launcher-385-thread-1) [n:127.0.0.1:55066_solr    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 855348 INFO  (jetty-launcher-385-thread-3) [n:127.0.0.1:55562_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 855348 INFO  (jetty-launcher-385-thread-1) [n:127.0.0.1:55066_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 855348 INFO  (jetty-launcher-385-thread-2) [n:127.0.0.1:48544_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 855350 INFO  (jetty-launcher-385-thread-1) [n:127.0.0.1:55066_solr    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94992915367526407-127.0.0.1:55066_solr-n_0000000000
   [junit4]   2> 855350 INFO  (jetty-launcher-385-thread-3) [n:127.0.0.1:55562_solr    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94992915367526409-127.0.0.1:55562_solr-n_0000000001
   [junit4]   2> 855350 INFO  (jetty-launcher-385-thread-2) [n:127.0.0.1:48544_solr    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94992915367526408-127.0.0.1:48544_solr-n_0000000002
   [junit4]   2> 855350 INFO  (jetty-launcher-385-thread-1) [n:127.0.0.1:55066_solr    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:55066_solr
   [junit4]   2> 855350 INFO  (jetty-launcher-385-thread-1) [n:127.0.0.1:55066_solr    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 855351 INFO  (jetty-launcher-385-thread-3) [n:127.0.0.1:55562_solr    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/94992915367526407-127.0.0.1:55066_solr-n_0000000000 to know if I could be the leader
   [junit4]   2> 855351 INFO  (jetty-launcher-385-thread-3) [n:127.0.0.1:55562_solr    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 855351 INFO  (jetty-launcher-385-thread-2) [n:127.0.0.1:48544_solr    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/94992915367526409-127.0.0.1:55562_solr-n_0000000001 to know if I could be the leader
   [junit4]   2> 855351 INFO  (jetty-launcher-385-thread-2) [n:127.0.0.1:48544_solr    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 855353 INFO  (jetty-launcher-385-thread-1) [n:127.0.0.1:55066_solr    ] o.a.s.c.Overseer Overseer (id=94992915367526407-127.0.0.1:55066_solr-n_0000000000) starting
   [junit4]   2> 855364 INFO  (jetty-launcher-385-thread-2) [n:127.0.0.1:48544_solr    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 855364 INFO  (jetty-launcher-385-thread-2) [n:127.0.0.1:48544_solr    ] o.a.s.c.CoreContainer Authentication plugin class obtained from system property 'authenticationPlugin': org.apache.solr.cloud.TestAuthenticationFramework$MockAuthenticationPlugin
   [junit4]   2> 855365 INFO  (jetty-launcher-385-thread-2) [n:127.0.0.1:48544_solr    ] o.a.s.c.CoreContainer Reconfiguring the shard handler factory and update shard handler.
   [junit4]   2> 855365 INFO  (jetty-launcher-385-thread-2) [n:127.0.0.1:48544_solr    ] o.a.s.h.c.HttpShardHandlerFactory Reconfiguring the default client with: org.apache.solr.cloud.TestAuthenticationFramework$MockAuthenticationPlugin$MockClientConfigurer@28ba9edb
   [junit4]   2> 855365 INFO  (jetty-launcher-385-thread-2) [n:127.0.0.1:48544_solr    ] o.a.s.u.UpdateShardHandler Reconfiguring the default client with: org.apache.solr.cloud.TestAuthenticationFramework$MockAuthenticationPlugin$MockClientConfigurer@28ba9edb
   [junit4]   2> 855366 INFO  (jetty-launcher-385-thread-2) [n:127.0.0.1:48544_solr    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestAuthenticationFramework_6B4A28DD8930C974-001/tempDir-001/node2/.
   [junit4]   2> 855366 INFO  (jetty-launcher-385-thread-2) [n:127.0.0.1:48544_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions
   [junit4]   2> 855374 INFO  (jetty-launcher-385-thread-2) [n:127.0.0.1:48544_solr    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 855374 INFO  (jetty-launcher-385-thread-2) [n:127.0.0.1:48544_solr    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 855376 INFO  (jetty-launcher-385-thread-1) [n:127.0.0.1:55066_solr    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 855381 INFO  (jetty-launcher-385-thread-3) [n:127.0.0.1:55562_solr    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 855381 INFO  (jetty-launcher-385-thread-3) [n:127.0.0.1:55562_solr    ] o.a.s.c.CoreContainer Authentication plugin class obtained from system property 'authenticationPlugin': org.apache.solr.cloud.TestAuthenticationFramework$MockAuthenticationPlugin
   [junit4]   2> 855382 INFO  (jetty-launcher-385-thread-3) [n:127.0.0.1:55562_solr    ] o.a.s.c.CoreContainer Reconfiguring the shard handler factory and update shard handler.
   [junit4]   2> 855382 INFO  (jetty-launcher-385-thread-3) [n:127.0.0.1:55562_solr    ] o.a.s.h.c.HttpShardHandlerFactory Reconfiguring the default client with: org.apache.solr.cloud.TestAuthenticationFramework$MockAuthenticationPlugin$MockClientConfigurer@404c9413
   [junit4]   2> 855382 INFO  (jetty-launcher-385-thread-3) [n:127.0.0.1:55562_solr    ] o.a.s.u.UpdateShardHandler Reconfiguring the default client with: org.apache.solr.cloud.TestAuthenticationFramework$MockAuthenticationPlugin$MockClientConfigurer@404c9413
   [junit4]   2> 855383 INFO  (jetty-launcher-385-thread-3) [n:127.0.0.1:55562_solr    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestAuthenticationFramework_6B4A28DD8930C974-001/tempDir-001/node3/.
   [junit4]   2> 855384 INFO  (jetty-launcher-385-thread-3) [n:127.0.0.1:55562_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions
   [junit4]   2> 855389 INFO  (jetty-launcher-385-thread-3) [n:127.0.0.1:55562_solr    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 855389 INFO  (jetty-launcher-385-thread-3) [n:127.0.0.1:55562_solr    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 855394 INFO  (jetty-launcher-385-thread-1) [n:127.0.0.1:55066_solr    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 855394 INFO  (OverseerCollectionConfigSetProcessor-94992915367526407-127.0.0.1:55066_solr-n_0000000000) [n:127.0.0.1:55066_solr    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 855394 INFO  (OverseerStateUpdate-94992915367526407-127.0.0.1:55066_solr-n_0000000000) [n:127.0.0.1:55066_solr    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 855395 INFO  (OverseerStateUpdate-94992915367526407-127.0.0.1:55066_solr-n_0000000000) [n:127.0.0.1:55066_solr    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 855404 INFO  (jetty-launcher-385-thread-1) [n:127.0.0.1:55066_solr    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 855430 INFO  (jetty-launcher-385-thread-1) [n:127.0.0.1:55066_solr    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 855430 INFO  (jetty-launcher-385-thread-1) [n:127.0.0.1:55066_solr    ] o.a.s.c.CoreContainer Authentication plugin class obtained from system property 'authenticationPlugin': org.apache.solr.cloud.TestAuthenticationFramework$MockAuthenticationPlugin
   [junit4]   2> 855430 INFO  (jetty-launcher-385-thread-1) [n:127.0.0.1:55066_solr    ] o.a.s.c.CoreContainer Reconfiguring the shard handler factory and update shard handler.
   [junit4]   2> 855431 INFO  (jetty-launcher-385-thread-1) [n:127.0.0.1:55066_solr    ] o.a.s.h.c.HttpShardHandlerFactory Reconfiguring the default client with: org.apache.solr.cloud.TestAuthenticationFramework$MockAuthenticationPlugin$MockClientConfigurer@40284289
   [junit4]   2> 855431 INFO  (jetty-launcher-385-thread-1) [n:127.0.0.1:55066_solr    ] o.a.s.u.UpdateShardHandler Reconfiguring the default client with: org.apache.solr.cloud.TestAuthenticationFramework$MockAuthenticationPlugin$MockClientConfigurer@40284289
   [junit4]   2> 855432 INFO  (jetty-launcher-385-thread-1) [n:127.0.0.1:55066_solr    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestAuthenticationFramework_6B4A28DD8930C974-001/tempDir-001/node1/.
   [junit4]   2> 855433 INFO  (jetty-launcher-385-thread-1) [n:127.0.0.1:55066_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions
   [junit4]   2> 855433 INFO  (jetty-launcher-385-thread-1) [n:127.0.0.1:55066_solr    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 855434 INFO  (jetty-launcher-385-thread-1) [n:127.0.0.1:55066_solr    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 855434 INFO  (TEST-TestAuthenticationFramework.testErrorsInShutdown-seed#[6B4A28DD8930C974]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 855434 INFO  (TEST-TestAuthenticationFramework.testErrorsInShutdown-seed#[6B4A28DD8930C974]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 855436 INFO  (zkCallback-399-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@4c79c234 name:ZooKeeperConnection Watcher:127.0.0.1:51098 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 855436 INFO  (TEST-TestAuthenticationFramework.testErrorsInShutdown-seed#[6B4A28DD8930C974]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 855437 INFO  (TEST-TestAuthenticationFramework.testErrorsInShutdown-seed#[6B4A28DD8930C974]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 855441 INFO  (jetty-launcher-385-thread-3) [    ] o.e.j.s.ServerConnector Stopped ServerConnector@c999844{HTTP/1.1}{127.0.0.1:0}
   [junit4]   2> 855441 INFO  (jetty-launcher-385-thread-3) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=54608992
   [junit4]   2> 855443 INFO  (zkCallback-396-thread-1-processing-n:127.0.0.1:55066_solr) [n:127.0.0.1:55066_solr    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 855444 INFO  (jetty-launcher-385-thread-2) [    ] o.e.j.s.ServerConnector Stopped ServerConnector@6d926c48{HTTP/1.1}{127.0.0.1:0}
   [junit4]   2> 855448 ERROR (jetty-launcher-385-thread-3-EventThread) [n:127.0.0.1:55562_solr    ] o.a.z.ClientCnxn Error while calling watcher 
   [junit4]   2> java.util.concurrent.RejectedExecutionException: Task org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1@d1129e1 rejected from org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor@61d4797a[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 1]
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2048)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:821)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1372)
   [junit4]   2> 	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.execute(ExecutorUtil.java:215)
   [junit4]   2> 	at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:110)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$3.process(SolrZkClient.java:261)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
   [junit4]   2> 855450 INFO  (jetty-launcher-385-thread-3) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@689a6a7e{/solr,null,UNAVAILABLE}
   [junit4]   2> 855449 INFO  (jetty-launcher-385-thread-2) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=547657681
   [junit4]   2> 855448 INFO  (zkCallback-396-thread-2-processing-n:127.0.0.1:55066_solr) [n:127.0.0.1:55066_solr    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 855451 INFO  (zkCallback-397-thread-1-processing-n:127.0.0.1:48544_solr) [n:127.0.0.1:48544_solr    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 855457 INFO  (jetty-launcher-385-thread-1) [    ] o.e.j.s.ServerConnector Stopped ServerConnector@65f9f092{HTTP/1.1}{127.0.0.1:0}
   [junit4]   2> 855460 INFO  (jetty-launcher-385-thread-2) [    ] o.a.s.c.Overseer Overseer (id=94992915367526407-127.0.0.1:55066_solr-n_0000000000) closing
   [junit4]   2> 855460 INFO  (OverseerStateUpdate-94992915367526407-127.0.0.1:55066_solr-n_0000000000) [n:127.0.0.1:55066_solr    ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:55066_solr
   [junit4]   2> 855460 INFO  (jetty-launcher-385-thread-1) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=399278526
   [junit4]   2> 855464 INFO  (zkCallback-397-thread-1-processing-n:127.0.0.1:48544_solr) [n:127.0.0.1:48544_solr    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 855464 WARN  (zkCallback-397-thread-2-processing-n:127.0.0.1:48544_solr) [n:127.0.0.1:48544_solr    ] o.a.s.c.LeaderElector Failed setting watch
   [junit4]   2> org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /overseer_elect/election/94992915367526407-127.0.0.1:55066_solr-n_0000000000
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:111)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:345)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:342)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:61)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:342)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:182)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.access$200(LeaderElector.java:59)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector$ElectionWatcher.process(LeaderElector.java:389)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$3$1.run(SolrZkClient.java:264)
   [junit4]   2> 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
   [junit4]   2> 	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
   [junit4]   2> 	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:232)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 855464 INFO  (zkCallback-396-thread-1-processing-n:127.0.0.1:55066_solr) [n:127.0.0.1:55066_solr    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 855465 INFO  (zkCallback-396-thread-2-processing-n:127.0.0.1:55066_solr) [n:127.0.0.1:55066_solr    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 855465 WARN  (zkCallback-396-thread-2-processing-n:127.0.0.1:55066_solr) [n:127.0.0.1:55066_solr    ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK
   [junit4]   2> 856964 WARN  (zkCallback-397-thread-1-processing-n:127.0.0.1:48544_solr) [n:127.0.0.1:48544_solr    ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK
   [junit4]   2> 856965 WARN  (zkCallback-396-thread-1-processing-n:127.0.0.1:55066_solr) [n:127.0.0.1:55066_solr    ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK
   [junit4]   2> 856966 WARN  (zkCallback-397-thread-2-processing-n:127.0.0.1:48544_solr) [n:127.0.0.1:48544_solr    ] o.a.s.c.LeaderElector 
   [junit4]   2> org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/election
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1472)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:328)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:325)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:61)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:325)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:108)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:190)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.access$200(LeaderElector.java:59)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector$ElectionWatcher.process(LeaderElector.java:389)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$3$1.run(SolrZkClient.java:264)
   [junit4]   2> 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
   [junit4]   2> 	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
   [junit4]   2> 	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:232)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 856966 INFO  (jetty-launcher-385-thread-2) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@37db9ed2{/solr,null,UNAVAILABLE}
   [junit4]   2> 856968 INFO  (jetty-launcher-385-thread-1) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@339809a4{/solr,null,UNAVAILABLE}
   [junit4]   2> 856972 INFO  (TEST-TestAuthenticationFramework.testErrorsInShutdown-seed#[6B4A28DD8930C974]) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:51098 51098
   [junit4]   2> 857194 INFO  (Thread-48265) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:51098 51098
   [junit4]   2> 857195 WARN  (Thread-48265) [    ] o.a.s.c.ZkTestServer Watch limit violations: 
   [junit4]   2> Maximum concurrent create/delete watches above limit:
   [junit4]   2> 
   [junit4]   2> 	4	/solr/security.json
   [junit4]   2> 	4	/solr/aliases.json
   [junit4]   2> 
   [junit4]   2> Maximum concurrent data watches above limit:
   [junit4]   2> 
   [junit4]   2> 	4	/solr/clusterstate.json
   [junit4]   2> 
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2> 
   [junit4]   2> 	4	/solr/collections
   [junit4]   2> 	4	/solr/live_nodes
   [junit4]   2> 
   [junit4]   2> 857202 INFO  (TEST-TestAuthenticationFramework.testCollectionCreateSearchDelete-seed#[6B4A28DD8930C974]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 857202 INFO  (Thread-48302) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 857202 INFO  (Thread-48302) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 857302 INFO  (TEST-TestAuthenticationFramework.testCollectionCreateSearchDelete-seed#[6B4A28DD8930C974]) [    ] o.a.s.c.ZkTestServer start zk server on port:45652
   [junit4]   2> 857302 INFO  (TEST-TestAuthenticationFramework.testCollectionCreateSearchDelete-seed#[6B4A28DD8930C974]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 857303 INFO  (TEST-TestAuthenticationFramework.testCollectionCreateSearchDelete-seed#[6B4A28DD8930C974]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 857310 INFO  (zkCallback-402-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1006d0ee name:ZooKeeperConnection Watcher:127.0.0.1:45652 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 857311 INFO  (TEST-TestAuthenticationFramework.testCollectionCreateSearchDelete-seed#[6B4A28DD8930C974]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 857311 INFO  (TEST-TestAuthenticationFramework.testCollectionCreateSearchDelete-seed#[6B4A28DD8930C974]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 857311 INFO  (TEST-TestAuthenticationFramework.testCollectionCreateSearchDelete-seed#[6B4A28DD8930C974]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr/solr.xml
   [junit4]   2> 857320 INFO  (jetty-launcher-401-thread-1) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 857320 INFO  (jetty-launcher-401-thread-5) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 857320 INFO  (jetty-launcher-401-thread-2) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 857322 INFO  (jetty-launcher-401-thread-4) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 857322 INFO  (jetty-launcher-401-thread-3) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 857324 INFO  (jetty-launcher-401-thread-1) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@50fcf69a{/solr,null,AVAILABLE}
   [junit4]   2> 857325 INFO  (jetty-launcher-401-thread-1) [    ] o.e.j.s.ServerConnector Started ServerConnector@4a64f75f{HTTP/1.1}{127.0.0.1:57569}
   [junit4]   2> 857325 INFO  (jetty-launcher-401-thread-1) [    ] o.e.j.s.Server Started @860217ms
   [junit4]   2> 857325 INFO  (jetty-launcher-401-thread-1) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=57569}
   [junit4]   2> 857325 INFO  (jetty-launcher-401-thread-1) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@3827de24
   [junit4]   2> 857325 INFO  (jetty-launcher-401-thread-1) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestAuthenticationFramework_6B4A28DD8930C974-001/tempDir-002/node1'
   [junit4]   2> 857325 INFO  (jetty-launcher-401-thread-1) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 857325 INFO  (jetty-launcher-401-thread-1) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 857326 INFO  (jetty-launcher-401-thread-5) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@6ea123a9{/solr,null,AVAILABLE}
   [junit4]   2> 857326 INFO  (jetty-launcher-401-thread-1) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 857326 INFO  (jetty-launcher-401-thread-5) [    ] o.e.j.s.ServerConnector Started ServerConnector@159a28e7{HTTP/1.1}{127.0.0.1:51048}
   [junit4]   2> 857326 INFO  (jetty-launcher-401-thread-1) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 857326 INFO  (jetty-launcher-401-thread-5) [    ] o.e.j.s.Server Started @860218ms
   [junit4]   2> 857326 INFO  (jetty-launcher-401-thread-5) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostPort=51048, hostContext=/solr}
   [junit4]   2> 857326 INFO  (jetty-launcher-401-thread-5) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@3827de24
   [junit4]   2> 857326 INFO  (jetty-launcher-401-thread-5) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestAuthenticationFramework_6B4A28DD8930C974-001/tempDir-002/node5'
   [junit4]   2> 857327 INFO  (jetty-launcher-401-thread-5) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 857327 INFO  (jetty-launcher-401-thread-5) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 857327 INFO  (jetty-launcher-401-thread-5) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 857327 INFO  (jetty-launcher-401-thread-5) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 857328 INFO  (jetty-launcher-401-thread-3) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@8d2b986{/solr,null,AVAILABLE}
   [junit4]   2> 857329 INFO  (jetty-launcher-401-thread-2) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@2875274a{/solr,null,AVAILABLE}
   [junit4]   2> 857329 INFO  (jetty-launcher-401-thread-2) [    ] o.e.j.s.ServerConnector Started ServerConnector@3ef03973{HTTP/1.1}{127.0.0.1:49406}
   [junit4]   2> 857329 INFO  (jetty-launcher-401-thread-2) [    ] o.e.j.s.Server Started @860221ms
   [junit4]   2> 857330 INFO  (jetty-launcher-401-thread-2) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=49406}
   [junit4]   2> 857330 INFO  (jetty-launcher-401-thread-2) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@3827de24
   [junit4]   2> 857330 INFO  (jetty-launcher-401-thread-2) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestAuthenticationFramework_6B4A28DD8930C974-001/tempDir-002/node2'
   [junit4]   2> 857330 INFO  (jetty-launcher-401-thread-2) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 857330 INFO  (jetty-launcher-401-thread-2) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 857330 INFO  (jetty-launcher-401-thread-2) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 857331 INFO  (zkCallback-404-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@14f71064 name:ZooKeeperConnection Watcher:127.0.0.1:45652/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 857331 INFO  (jetty-launcher-401-thread-5) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 857331 INFO  (jetty-launcher-401-thread-5) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 857331 INFO  (zkCallback-403-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7d3420 name:ZooKeeperConnection Watcher:127.0.0.1:45652/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 857331 INFO  (jetty-launcher-401-thread-1) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 857332 INFO  (jetty-launcher-401-thread-5) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 857332 INFO  (jetty-launcher-401-thread-1) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 857332 INFO  (jetty-launcher-401-thread-1) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 857332 INFO  (jetty-launcher-401-thread-2) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 857333 INFO  (jetty-launcher-401-thread-3) [    ] o.e.j.s.ServerConnector Started ServerConnector@3a3a0c92{HTTP/1.1}{127.0.0.1:57931}
   [junit4]   2> 857333 INFO  (jetty-launcher-401-thread-3) [    ] o.e.j.s.Server Started @860225ms
   [junit4]   2> 857333 INFO  (jetty-launcher-401-thread-3) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=57931}
   [junit4]   2> 857333 INFO  (jetty-launcher-401-thread-3) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@3827de24
   [junit4]   2> 857333 INFO  (jetty-launcher-401-thread-3) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestAuthenticationFramework_6B4A28DD8930C974-001/tempDir-002/node3'
   [junit4]   2> 857333 INFO  (jetty-launcher-401-thread-3) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 857333 INFO  (jetty-launcher-401-thread-3) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 857333 INFO  (jetty-launcher-401-thread-3) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 857334 INFO  (jetty-launcher-401-thread-3) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 857336 INFO  (jetty-launcher-401-thread-4) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@54d573b6{/solr,null,AVAILABLE}
   [junit4]   2> 857337 INFO  (jetty-launcher-401-thread-4) [    ] o.e.j.s.ServerConnector Started ServerConnector@99fdd3d{HTTP/1.1}{127.0.0.1:44879}
   [junit4]   2> 857337 INFO  (jetty-launcher-401-thread-4) [    ] o.e.j.s.Server Started @860229ms
   [junit4]   2> 857337 INFO  (jetty-launcher-401-thread-4) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=44879}
   [junit4]   2> 857337 INFO  (jetty-launcher-401-thread-4) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@3827de24
   [junit4]   2> 857337 INFO  (jetty-launcher-401-thread-4) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestAuthenticationFramework_6B4A28DD8930C974-001/tempDir-002/node4'
   [junit4]   2> 857337 INFO  (jetty-launcher-401-thread-4) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 857337 INFO  (jetty-launcher-401-thread-4) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 857337 INFO  (jetty-launcher-401-thread-4) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 857338 INFO  (zkCallback-405-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@177e48ed name:ZooKeeperConnection Watcher:127.0.0.1:45652/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 857338 INFO  (jetty-launcher-401-thread-2) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 857338 INFO  (jetty-launcher-401-thread-2) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 857339 INFO  (zkCallback-406-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5abcee89 name:ZooKeeperConnection Watcher:127.0.0.1:45652/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 857339 INFO  (jetty-launcher-401-thread-3) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 857339 INFO  (jetty-launcher-401-thread-2) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 857339 INFO  (jetty-launcher-401-thread-3) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 857339 INFO  (jetty-launcher-401-thread-3) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 857344 INFO  (jetty-launcher-401-thread-4) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 857345 INFO  (zkCallback-407-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1beffb48 name:ZooKeeperConnection Watcher:127.0.0.1:45652/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 857346 INFO  (jetty-launcher-401-thread-4) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 857346 INFO  (jetty-launcher-401-thread-4) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 857346 INFO  (jetty-launcher-401-thread-4) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 857351 INFO  (jetty-launcher-401-thread-1) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestAuthenticationFramework_6B4A28DD8930C974-001/tempDir-002/node1/.
   [junit4]   2> 857352 INFO  (jetty-launcher-401-thread-5) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestAuthenticationFramework_6B4A28DD8930C974-001/tempDir-002/node5/.
   [junit4]   2> 857352 INFO  (jetty-launcher-401-thread-5) [    ] o.a.s.c.CoreContainer New CoreContainer 844108746
   [junit4]   2> 857352 INFO  (jetty-launcher-401-thread-5) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestAuthenticationFramework_6B4A28DD8930C974-001/tempDir-002/node5]
   [junit4]   2> 857352 WARN  (jetty-launcher-401-thread-5) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestAuthenticationFramework_6B4A28DD8930C974-001/tempDir-002/node5/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestAuthenticationFramework_6B4A28DD8930C974-001/tempDir-002/node5/lib
   [junit4]   2> 857353 INFO  (jetty-launcher-401-thread-5) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 857354 INFO  (jetty-launcher-401-thread-5) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 857355 INFO  (jetty-launcher-401-thread-5) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 857355 INFO  (jetty-launcher-401-thread-5) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 857355 INFO  (jetty-launcher-401-thread-5) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:45652/solr
   [junit4]   2> 857355 INFO  (jetty-launcher-401-thread-5) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 857355 INFO  (jetty-launcher-401-thread-5) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 857355 INFO  (jetty-launcher-401-thread-1) [    ] o.a.s.c.CoreContainer New CoreContainer 626263853
   [junit4]   2> 857356 INFO  (jetty-launcher-401-thread-5) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 857360 INFO  (jetty-launcher-401-thread-1) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestAuthenticationFramework_6B4A28DD8930C974-001/tempDir-002/node1]
   [junit4]   2> 857361 WARN  (jetty-launcher-401-thread-1) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestAuthenticationFramework_6B4A28DD8930C974-001/tempDir-002/node1/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Nigh

[...truncated too long message...]

0(QueuedThreadPool.java:47)
   [junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >    5) Thread[id=50658, name=qtp1598845069-50658, state=TIMED_WAITING, group=TGRP-TestAuthenticationFramework]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
   [junit4]    >         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)
   [junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)
   [junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)
   [junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >    6) Thread[id=50654, name=qtp1598845069-50654, state=TIMED_WAITING, group=TGRP-TestAuthenticationFramework]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
   [junit4]    >         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)
   [junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)
   [junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)
   [junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >    7) Thread[id=50660, name=qtp1598845069-50660, state=TIMED_WAITING, group=TGRP-TestAuthenticationFramework]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
   [junit4]    >         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)
   [junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)
   [junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)
   [junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >    8) Thread[id=50653, name=qtp1598845069-50653, state=TIMED_WAITING, group=TGRP-TestAuthenticationFramework]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
   [junit4]    >         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)
   [junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)
   [junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)
   [junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >    9) Thread[id=50659, name=qtp1598845069-50659, state=TIMED_WAITING, group=TGRP-TestAuthenticationFramework]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
   [junit4]    >         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)
   [junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)
   [junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)
   [junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >   10) Thread[id=50662, name=org.eclipse.jetty.server.session.HashSessionManager@25ab2976Timer, state=TIMED_WAITING, group=TGRP-TestAuthenticationFramework]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
   [junit4]    >         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1090)
   [junit4]    >         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([6B4A28DD8930C974]:0)Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   [junit4]    >    1) Thread[id=50661, name=Scheduler-304156586, state=TIMED_WAITING, group=TGRP-TestAuthenticationFramework]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
   [junit4]    >         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1090)
   [junit4]    >         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >    2) Thread[id=50662, name=org.eclipse.jetty.server.session.HashSessionManager@25ab2976Timer, state=TIMED_WAITING, group=TGRP-TestAuthenticationFramework]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
   [junit4]    >         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1090)
   [junit4]    >         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([6B4A28DD8930C974]:0)
   [junit4] Completed [131/553] on J2 in 123.11s, 7 tests, 3 errors <<< FAILURES!

[...truncated 1295 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:799: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:736: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 553 suites (7 ignored), 2187 tests, 2 suite-level errors, 1 error, 1252 ignored (15 assumptions) [seed: 6B4A28DD8930C974]

Total time: 231 minutes 4 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1038 - Still Failing

[Apache Jenkins Server <je...@builds.apache.org>]

Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1038/

2 tests failed.
FAILED:  org.apache.solr.handler.TestReplicationHandler.doTestReplicateAfterCoreReload

Error Message:
expected:<[{indexVersion=1449417291014,generation=2,filelist=[_0.fdt, _0.fdx, _0.fnm, _0.nvd, _0.nvm, _0.si, _0_Lucene50_0.doc, _0_Lucene50_0.tim, _0_Lucene50_0.tip, _1.fdt, _1.fdx, _1.fnm, _1.nvd, _1.nvm, _1.si, _1_Lucene50_0.doc, _1_Lucene50_0.tim, _1_Lucene50_0.tip, _2.fdt, _2.fdx, _2.fnm, _2.nvd, _2.nvm, _2.si, _2_Lucene50_0.doc, _2_Lucene50_0.tim, _2_Lucene50_0.tip, _3.fdt, _3.fdx, _3.fnm, _3.nvd, _3.nvm, _3.si, _3_Lucene50_0.doc, _3_Lucene50_0.tim, _3_Lucene50_0.tip, _4.fdt, _4.fdx, _4.fnm, _4.nvd, _4.nvm, _4.si, _4_Lucene50_0.doc, _4_Lucene50_0.tim, _4_Lucene50_0.tip, _5.fdt, _5.fdx, _5.fnm, _5.nvd, _5.nvm, _5.si, _5_Lucene50_0.doc, _5_Lucene50_0.tim, _5_Lucene50_0.tip, segments_2]}]> but was:<[{indexVersion=1449417291014,generation=3,filelist=[_6.cfe, _6.cfs, _6.si, segments_3]}, {indexVersion=1449417291014,generation=2,filelist=[_0.fdt, _0.fdx, _0.fnm, _0.nvd, _0.nvm, _0.si, _0_Lucene50_0.doc, _0_Lucene50_0.tim, _0_Lucene50_0.tip, _1.fdt, _1.fdx, _1.fnm, _1.nvd, _1.nvm, _1.si, _1_Lucene50_0.doc, _1_Lucene50_0.tim, _1_Lucene50_0.tip, _2.fdt, _2.fdx, _2.fnm, _2.nvd, _2.nvm, _2.si, _2_Lucene50_0.doc, _2_Lucene50_0.tim, _2_Lucene50_0.tip, _3.fdt, _3.fdx, _3.fnm, _3.nvd, _3.nvm, _3.si, _3_Lucene50_0.doc, _3_Lucene50_0.tim, _3_Lucene50_0.tip, _4.fdt, _4.fdx, _4.fnm, _4.nvd, _4.nvm, _4.si, _4_Lucene50_0.doc, _4_Lucene50_0.tim, _4_Lucene50_0.tip, _5.fdt, _5.fdx, _5.fnm, _5.nvd, _5.nvm, _5.si, _5_Lucene50_0.doc, _5_Lucene50_0.tim, _5_Lucene50_0.tip, segments_2]}]>

Stack Trace:
java.lang.AssertionError: expected:<[{indexVersion=1449417291014,generation=2,filelist=[_0.fdt, _0.fdx, _0.fnm, _0.nvd, _0.nvm, _0.si, _0_Lucene50_0.doc, _0_Lucene50_0.tim, _0_Lucene50_0.tip, _1.fdt, _1.fdx, _1.fnm, _1.nvd, _1.nvm, _1.si, _1_Lucene50_0.doc, _1_Lucene50_0.tim, _1_Lucene50_0.tip, _2.fdt, _2.fdx, _2.fnm, _2.nvd, _2.nvm, _2.si, _2_Lucene50_0.doc, _2_Lucene50_0.tim, _2_Lucene50_0.tip, _3.fdt, _3.fdx, _3.fnm, _3.nvd, _3.nvm, _3.si, _3_Lucene50_0.doc, _3_Lucene50_0.tim, _3_Lucene50_0.tip, _4.fdt, _4.fdx, _4.fnm, _4.nvd, _4.nvm, _4.si, _4_Lucene50_0.doc, _4_Lucene50_0.tim, _4_Lucene50_0.tip, _5.fdt, _5.fdx, _5.fnm, _5.nvd, _5.nvm, _5.si, _5_Lucene50_0.doc, _5_Lucene50_0.tim, _5_Lucene50_0.tip, segments_2]}]> but was:<[{indexVersion=1449417291014,generation=3,filelist=[_6.cfe, _6.cfs, _6.si, segments_3]}, {indexVersion=1449417291014,generation=2,filelist=[_0.fdt, _0.fdx, _0.fnm, _0.nvd, _0.nvm, _0.si, _0_Lucene50_0.doc, _0_Lucene50_0.tim, _0_Lucene50_0.tip, _1.fdt, _1.fdx, _1.fnm, _1.nvd, _1.nvm, _1.si, _1_Lucene50_0.doc, _1_Lucene50_0.tim, _1_Lucene50_0.tip, _2.fdt, _2.fdx, _2.fnm, _2.nvd, _2.nvm, _2.si, _2_Lucene50_0.doc, _2_Lucene50_0.tim, _2_Lucene50_0.tip, _3.fdt, _3.fdx, _3.fnm, _3.nvd, _3.nvm, _3.si, _3_Lucene50_0.doc, _3_Lucene50_0.tim, _3_Lucene50_0.tip, _4.fdt, _4.fdx, _4.fnm, _4.nvd, _4.nvm, _4.si, _4_Lucene50_0.doc, _4_Lucene50_0.tim, _4_Lucene50_0.tip, _5.fdt, _5.fdx, _5.fnm, _5.nvd, _5.nvm, _5.si, _5_Lucene50_0.doc, _5_Lucene50_0.tim, _5_Lucene50_0.tip, segments_2]}]>
	at __randomizedtesting.SeedInfo.seed([DA5873B6F3FA9AF8:FF8F688683B294FB]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.failNotEquals(Assert.java:647)
	at org.junit.Assert.assertEquals(Assert.java:128)
	at org.junit.Assert.assertEquals(Assert.java:147)
	at org.apache.solr.handler.TestReplicationHandler.doTestReplicateAfterCoreReload(TestReplicationHandler.java:1140)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.handler.TestReplicationHandler.doTestStressReplication

Error Message:
timed out waiting for collection1 startAt time to exceed: Mon Dec 07 02:56:23 SBT 2015

Stack Trace:
java.lang.AssertionError: timed out waiting for collection1 startAt time to exceed: Mon Dec 07 02:56:23 SBT 2015
	at __randomizedtesting.SeedInfo.seed([DA5873B6F3FA9AF8:1F37370F6D2F34B]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.solr.handler.TestReplicationHandler.watchCoreStartAt(TestReplicationHandler.java:1419)
	at org.apache.solr.handler.TestReplicationHandler.doTestStressReplication(TestReplicationHandler.java:771)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 10731 lines...]
   [junit4] Suite: org.apache.solr.handler.TestReplicationHandler
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/init-core-data-001
   [junit4]   2> 1225543 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.SolrTestCaseJ4 ###Starting doTestRepeater
   [junit4]   2> 1225544 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-001/collection1
   [junit4]   2> 1225580 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 1225637 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@108ae9d5{/solr,null,AVAILABLE}
   [junit4]   2> 1225654 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.e.j.s.ServerConnector Started ServerConnector@55fb709c{HTTP/1.1}{127.0.0.1:34195}
   [junit4]   2> 1225654 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.e.j.s.Server Started @1228780ms
   [junit4]   2> 1225654 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostPort=34195, hostContext=/solr, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-001/collection1/data}
   [junit4]   2> 1225654 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@4f6de641
   [junit4]   2> 1225654 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-001'
   [junit4]   2> 1225655 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1225655 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1225655 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-001/solr.xml
   [junit4]   2> 1225699 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-001/.
   [junit4]   2> 1225699 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.CoreContainer New CoreContainer 1822242929
   [junit4]   2> 1225699 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-001]
   [junit4]   2> 1225700 WARN  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-001/lib
   [junit4]   2> 1225701 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1225703 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1225704 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1225704 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1225705 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1225705 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1225707 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-001/.
   [junit4]   2> 1225715 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {name=collection1, transient=false, loadOnStartup=true, config=solrconfig.xml, dataDir=data/, configSetProperties=configsetprops.json, schema=schema.xml}
   [junit4]   2> 1225716 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-001/./collection1
   [junit4]   2> 1225716 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 1225727 INFO  (coreLoadExecutor-4439-thread-1) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-001/collection1'
   [junit4]   2> 1225738 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0
   [junit4]   2> 1225744 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1225744 INFO  (coreLoadExecutor-4439-thread-1) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1225744 INFO  (coreLoadExecutor-4439-thread-1) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1225753 INFO  (coreLoadExecutor-4439-thread-1) [    ] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 1225766 INFO  (coreLoadExecutor-4439-thread-1) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.5.0
   [junit4]   2> 1225790 INFO  (coreLoadExecutor-4439-thread-1) [    ] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1225795 INFO  (coreLoadExecutor-4439-thread-1) [    ] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1225814 INFO  (coreLoadExecutor-4439-thread-1) [    ] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 1225817 INFO  (coreLoadExecutor-4439-thread-1) [    ] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-001/collection1'
   [junit4]   2> 1225817 INFO  (coreLoadExecutor-4439-thread-1) [    ] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-001/./collection1
   [junit4]   2> 1225817 INFO  (coreLoadExecutor-4439-thread-1) [    x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 1225818 INFO  (coreLoadExecutor-4439-thread-1) [    x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-001/collection1], dataDir=[null]
   [junit4]   2> 1225818 INFO  (coreLoadExecutor-4439-thread-1) [    x:collection1] o.a.s.c.SolrCore JMX monitoring not detected for core: collection1
   [junit4]   2> 1225818 INFO  (coreLoadExecutor-4439-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-001/./collection1/data
   [junit4]   2> 1225819 INFO  (coreLoadExecutor-4439-thread-1) [    x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-001/./collection1/data/index/
   [junit4]   2> 1225819 WARN  (coreLoadExecutor-4439-thread-1) [    x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-001/./collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 1225819 INFO  (coreLoadExecutor-4439-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-001/./collection1/data/index
   [junit4]   2> 1225819 INFO  (coreLoadExecutor-4439-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=19, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.36844622650321657]
   [junit4]   2> 1225820 INFO  (coreLoadExecutor-4439-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=RawDirectoryWrapper(RAMDirectory@569f4e3e lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@74c9f4df),segFN=segments_1,generation=1}
   [junit4]   2> 1225820 INFO  (coreLoadExecutor-4439-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 1225845 INFO  (coreLoadExecutor-4439-thread-1) [    x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 1225848 INFO  (coreLoadExecutor-4439-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1225849 INFO  (coreLoadExecutor-4439-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1225850 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [    x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-001/./collection1/data/
   [junit4]   2> 1225851 INFO  (coreLoadExecutor-4439-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1225852 INFO  (coreLoadExecutor-4439-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1225858 INFO  (coreLoadExecutor-4439-thread-1) [    x:collection1] o.a.s.c.PluginBag requestHandler : 'lazy' created with startup=lazy 
   [junit4]   2> 1225858 INFO  (coreLoadExecutor-4439-thread-1) [    x:collection1] o.a.s.c.RequestHandlers Registered paths: lazy,/admin/ping,/admin/properties,/update,/admin/luke,/admin/system,/admin/mbeans,/config,/update/json/docs,/admin/threads,/admin/plugins,/replication,/schema,/get,/update/json,standard,/admin/file,/update/csv,defaults,/admin/logging,/admin/segments
   [junit4]   2> 1225863 INFO  (coreLoadExecutor-4439-thread-1) [    x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 1225864 INFO  (coreLoadExecutor-4439-thread-1) [    x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1225864 INFO  (coreLoadExecutor-4439-thread-1) [    x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1225866 INFO  (coreLoadExecutor-4439-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@790ec6e7
   [junit4]   2> 1225866 INFO  (coreLoadExecutor-4439-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=RawDirectoryWrapper(RAMDirectory@569f4e3e lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@74c9f4df),segFN=segments_1,generation=1}
   [junit4]   2> 1225866 INFO  (coreLoadExecutor-4439-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 1225867 INFO  (coreLoadExecutor-4439-thread-1) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@15000ac7[collection1] main
   [junit4]   2> 1225867 INFO  (coreLoadExecutor-4439-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-001/collection1/conf
   [junit4]   2> 1225867 INFO  (coreLoadExecutor-4439-thread-1) [    x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {storageDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-001/collection1/conf}
   [junit4]   2> 1225867 INFO  (coreLoadExecutor-4439-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using file:dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-001/collection1/conf
   [junit4]   2> 1225867 INFO  (coreLoadExecutor-4439-thread-1) [    x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 1225868 INFO  (coreLoadExecutor-4439-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler Replication enabled for following config files: schema.xml,xslt/dummy.xsl
   [junit4]   2> 1225868 INFO  (coreLoadExecutor-4439-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1225868 INFO  (coreLoadExecutor-4439-thread-1) [    x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 1225880 INFO  (searcherExecutor-4440-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@15000ac7[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1226246 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-002/collection1
   [junit4]   2> 1226249 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 1226271 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@a046505{/solr,null,AVAILABLE}
   [junit4]   2> 1226272 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.e.j.s.ServerConnector Started ServerConnector@26843066{HTTP/1.1}{127.0.0.1:39777}
   [junit4]   2> 1226272 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.e.j.s.Server Started @1229398ms
   [junit4]   2> 1226272 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-002/collection1/data, hostPort=39777, hostContext=/solr}
   [junit4]   2> 1226272 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@4f6de641
   [junit4]   2> 1226272 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-002'
   [junit4]   2> 1226272 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1226272 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1226273 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-002/solr.xml
   [junit4]   2> 1226317 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-002/.
   [junit4]   2> 1226318 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.CoreContainer New CoreContainer 443998018
   [junit4]   2> 1226318 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-002]
   [junit4]   2> 1226318 WARN  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-002/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-002/lib
   [junit4]   2> 1226321 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1226326 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1226327 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1226327 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1226327 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1226327 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1226328 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-002/.
   [junit4]   2> 1226338 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {name=collection1, dataDir=data/, loadOnStartup=true, transient=false, configSetProperties=configsetprops.json, config=solrconfig.xml, schema=schema.xml}
   [junit4]   2> 1226339 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-002/./collection1
   [junit4]   2> 1226339 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 1226358 INFO  (coreLoadExecutor-4449-thread-1) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-002/collection1'
   [junit4]   2> 1226359 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0
   [junit4]   2> 1226359 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1226359 INFO  (coreLoadExecutor-4449-thread-1) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1226359 INFO  (coreLoadExecutor-4449-thread-1) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1226367 INFO  (coreLoadExecutor-4449-thread-1) [    ] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 1226494 INFO  (coreLoadExecutor-4449-thread-1) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.5.0
   [junit4]   2> 1226584 INFO  (coreLoadExecutor-4449-thread-1) [    ] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1226600 INFO  (coreLoadExecutor-4449-thread-1) [    ] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1226617 INFO  (coreLoadExecutor-4449-thread-1) [    ] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 1226621 INFO  (coreLoadExecutor-4449-thread-1) [    ] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-002/collection1'
   [junit4]   2> 1226621 INFO  (coreLoadExecutor-4449-thread-1) [    ] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-002/./collection1
   [junit4]   2> 1226621 INFO  (coreLoadExecutor-4449-thread-1) [    x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 1226622 INFO  (coreLoadExecutor-4449-thread-1) [    x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-002/collection1], dataDir=[null]
   [junit4]   2> 1226622 INFO  (coreLoadExecutor-4449-thread-1) [    x:collection1] o.a.s.c.SolrCore JMX monitoring not detected for core: collection1
   [junit4]   2> 1226625 INFO  (coreLoadExecutor-4449-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-002/./collection1/data
   [junit4]   2> 1226625 INFO  (coreLoadExecutor-4449-thread-1) [    x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-002/./collection1/data/index/
   [junit4]   2> 1226625 WARN  (coreLoadExecutor-4449-thread-1) [    x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-002/./collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 1226626 INFO  (coreLoadExecutor-4449-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-002/./collection1/data/index
   [junit4]   2> 1226626 INFO  (coreLoadExecutor-4449-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=19, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.36844622650321657]
   [junit4]   2> 1226629 INFO  (coreLoadExecutor-4449-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=RawDirectoryWrapper(RAMDirectory@19dd431f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@d61b57d),segFN=segments_1,generation=1}
   [junit4]   2> 1226629 INFO  (coreLoadExecutor-4449-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 1226643 INFO  (coreLoadExecutor-4449-thread-1) [    x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 1226645 INFO  (coreLoadExecutor-4449-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1226646 INFO  (coreLoadExecutor-4449-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1226647 INFO  (coreLoadExecutor-4449-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1226648 INFO  (coreLoadExecutor-4449-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1226649 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [    x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-002/./collection1/data/
   [junit4]   2> 1226659 INFO  (coreLoadExecutor-4449-thread-1) [    x:collection1] o.a.s.c.PluginBag requestHandler : 'lazy' created with startup=lazy 
   [junit4]   2> 1226659 INFO  (coreLoadExecutor-4449-thread-1) [    x:collection1] o.a.s.c.RequestHandlers Registered paths: lazy,/admin/ping,/admin/properties,/update,/admin/luke,/admin/system,/admin/mbeans,/config,/update/json/docs,/admin/threads,/admin/plugins,/replication,/schema,/get,/update/json,standard,/admin/file,/update/csv,defaults,/admin/logging,/admin/segments
   [junit4]   2> 1226660 INFO  (coreLoadExecutor-4449-thread-1) [    x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 1226667 INFO  (coreLoadExecutor-4449-thread-1) [    x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1226667 INFO  (coreLoadExecutor-4449-thread-1) [    x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1226667 INFO  (coreLoadExecutor-4449-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@4746c9ee
   [junit4]   2> 1226668 INFO  (coreLoadExecutor-4449-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=RawDirectoryWrapper(RAMDirectory@19dd431f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@d61b57d),segFN=segments_1,generation=1}
   [junit4]   2> 1226668 INFO  (coreLoadExecutor-4449-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 1226668 INFO  (coreLoadExecutor-4449-thread-1) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@59b82977[collection1] main
   [junit4]   2> 1226668 INFO  (coreLoadExecutor-4449-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-002/collection1/conf
   [junit4]   2> 1226668 INFO  (coreLoadExecutor-4449-thread-1) [    x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {storageDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-002/collection1/conf}
   [junit4]   2> 1226668 INFO  (coreLoadExecutor-4449-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using file:dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-002/collection1/conf
   [junit4]   2> 1226669 INFO  (coreLoadExecutor-4449-thread-1) [    x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 1226671 INFO  (coreLoadExecutor-4449-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler Poll scheduled at an interval of 1000ms
   [junit4]   2> 1226671 INFO  (coreLoadExecutor-4449-thread-1) [    x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 1226672 INFO  (searcherExecutor-4450-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@59b82977[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1226866 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.e.j.s.ServerConnector Stopped ServerConnector@26843066{HTTP/1.1}{127.0.0.1:0}
   [junit4]   2> 1226866 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=443998018
   [junit4]   2> 1226866 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    x:collection1] o.a.s.c.SolrCore [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@661bb296
   [junit4]   2> 1226867 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    x:collection1] o.a.s.u.DirectUpdateHandler2 closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
   [junit4]   2> 1226867 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    x:collection1] o.a.s.u.SolrCoreState Closing SolrCoreState
   [junit4]   2> 1226867 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    x:collection1] o.a.s.u.DefaultSolrCoreState SolrCoreState ref count has reached 0 - closing IndexWriter
   [junit4]   2> 1226867 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    x:collection1] o.a.s.u.DefaultSolrCoreState closing IndexWriter with IndexWriterCloser
   [junit4]   2> 1226870 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    x:collection1] o.a.s.c.SolrCore [collection1] Closing main searcher on request.
   [junit4]   2> 1226870 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing MockDirectoryFactory - 2 directories currently being tracked
   [junit4]   2> 1226870 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    x:collection1] o.a.s.c.CachingDirectoryFactory looking to close /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-002/./collection1/data/index [CachedDir<<refCount=0;path=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-002/./collection1/data/index;done=false>>]
   [junit4]   2> 1226870 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-002/./collection1/data/index
   [junit4]   2> 1226870 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    x:collection1] o.a.s.c.CachingDirectoryFactory looking to close /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-002/./collection1/data [CachedDir<<refCount=0;path=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-002/./collection1/data;done=false>>]
   [junit4]   2> 1226871 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-002/./collection1/data
   [junit4]   2> 1226872 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    x:collection1] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@a046505{/solr,null,UNAVAILABLE}
   [junit4]   2> 1226874 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 1226877 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@b6329e9{/solr,null,AVAILABLE}
   [junit4]   2> 1226878 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.e.j.s.ServerConnector Started ServerConnector@1b5485d6{HTTP/1.1}{127.0.0.1:32879}
   [junit4]   2> 1226878 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.e.j.s.Server Started @1230004ms
   [junit4]   2> 1226878 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostPort=32879, hostContext=/solr, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-002/collection1/data}
   [junit4]   2> 1226878 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@4f6de641
   [junit4]   2> 1226878 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-002'
   [junit4]   2> 1226878 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1226879 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1226879 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-002/solr.xml
   [junit4]   2> 1226890 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-002/.
   [junit4]   2> 1226890 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.CoreContainer New CoreContainer 1045830494
   [junit4]   2> 1226890 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-002]
   [junit4]   2> 1226891 WARN  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-002/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-002/lib
   [junit4]   2> 1226892 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1226893 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1226893 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1226893 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1226893 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1226893 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1226894 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-002/.
   [junit4]   2> 1226895 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {transient=false, config=solrconfig.xml, dataDir=data/, name=collection1, configSetProperties=configsetprops.json, schema=schema.xml, loadOnStartup=true}
   [junit4]   2> 1226895 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-002/./collection1
   [junit4]   2> 1226895 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 1226895 INFO  (coreLoadExecutor-4460-thread-1) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-002/collection1'
   [junit4]   2> 1226897 INFO  (coreLoadExecutor-4460-thread-1) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1226897 INFO  (coreLoadExecutor-4460-thread-1) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1226905 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0
   [junit4]   2> 1226905 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1226907 INFO  (coreLoadExecutor-4460-thread-1) [    ] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 1226920 INFO  (coreLoadExecutor-4460-thread-1) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.5.0
   [junit4]   2> 1226944 INFO  (coreLoadExecutor-4460-thread-1) [    ] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1226949 INFO  (coreLoadExecutor-4460-thread-1) [    ] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1226956 INFO  (coreLoadExecutor-4460-thread-1) [    ] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 1226958 INFO  (coreLoadExecutor-4460-thread-1) [    ] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-002/collection1'
   [junit4]   2> 1226958 INFO  (coreLoadExecutor-4460-thread-1) [    ] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-002/./collection1
   [junit4]   2> 1226959 INFO  (coreLoadExecutor-4460-thread-1) [    x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 1226959 INFO  (coreLoadExecutor-4460-thread-1) [    x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-002/collection1], dataDir=[null]
   [junit4]   2> 1226959 INFO  (coreLoadExecutor-4460-thread-1) [    x:collection1] o.a.s.c.SolrCore JMX monitoring not detected for core: collection1
   [junit4]   2> 1226960 INFO  (coreLoadExecutor-4460-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-002/./collection1/data
   [junit4]   2> 1226960 INFO  (coreLoadExecutor-4460-thread-1) [    x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-002/./collection1/data/index/
   [junit4]   2> 1226960 WARN  (coreLoadExecutor-4460-thread-1) [    x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-002/./collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 1226960 INFO  (coreLoadExecutor-4460-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-002/./collection1/data/index
   [junit4]   2> 1226960 INFO  (coreLoadExecutor-4460-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=19, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.36844622650321657]
   [junit4]   2> 1226961 INFO  (coreLoadExecutor-4460-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=RawDirectoryWrapper(RAMDirectory@245d75ca lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@73e0336c),segFN=segments_1,generation=1}
   [junit4]   2> 1226961 INFO  (coreLoadExecutor-4460-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 1226963 INFO  (coreLoadExecutor-4460-thread-1) [    x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 1226966 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [    x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-002/./collection1/data/
   [junit4]   2> 1226966 INFO  (coreLoadExecutor-4460-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1226967 INFO  (coreLoadExecutor-4460-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1226969 INFO  (coreLoadExecutor-4460-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1226970 INFO  (coreLoadExecutor-4460-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1226971 INFO  (coreLoadExecutor-4460-thread-1) [    x:collection1] o.a.s.c.PluginBag requestHandler : 'lazy' created with startup=lazy 
   [junit4]   2> 1226971 INFO  (coreLoadExecutor-4460-thread-1) [    x:collection1] o.a.s.c.RequestHandlers Registered paths: lazy,/admin/ping,/admin/properties,/update,/admin/luke,/admin/system,/admin/mbeans,/config,/update/json/docs,/admin/threads,/admin/plugins,/replication,/schema,/get,/update/json,standard,/admin/file,/update/csv,defaults,/admin/logging,/admin/segments
   [junit4]   2> 1226972 INFO  (coreLoadExecutor-4460-thread-1) [    x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 1226973 INFO  (coreLoadExecutor-4460-thread-1) [    x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1226973 INFO  (coreLoadExecutor-4460-thread-1) [    x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1226973 INFO  (coreLoadExecutor-4460-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@7d1798fb
   [junit4]   2> 1226974 INFO  (coreLoadExecutor-4460-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=RawDirectoryWrapper(RAMDirectory@245d75ca lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@73e0336c),segFN=segments_1,generation=1}
   [junit4]   2> 1226974 INFO  (coreLoadExecutor-4460-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 1226974 INFO  (coreLoadExecutor-4460-thread-1) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@3e263278[collection1] main
   [junit4]   2> 1226974 INFO  (coreLoadExecutor-4460-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-002/collection1/conf
   [junit4]   2> 1226974 INFO  (coreLoadExecutor-4460-thread-1) [    x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {storageDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-002/collection1/conf}
   [junit4]   2> 1226974 INFO  (coreLoadExecutor-4460-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using file:dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-002/collection1/conf
   [junit4]   2> 1226974 INFO  (coreLoadExecutor-4460-thread-1) [    x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 1226974 INFO  (coreLoadExecutor-4460-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1226974 INFO  (coreLoadExecutor-4460-thread-1) [    x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 1226975 INFO  (searcherExecutor-4461-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@3e263278[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1227406 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-003/collection1
   [junit4]   2> 1227411 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 1227418 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@34d9a745{/solr,null,AVAILABLE}
   [junit4]   2> 1227418 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.e.j.s.ServerConnector Started ServerConnector@3d74be1e{HTTP/1.1}{127.0.0.1:49434}
   [junit4]   2> 1227418 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.e.j.s.Server Started @1230544ms
   [junit4]   2> 1227418 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-003/collection1/data, hostContext=/solr, hostPort=49434}
   [junit4]   2> 1227419 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@4f6de641
   [junit4]   2> 1227419 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-003'
   [junit4]   2> 1227419 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1227419 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1227419 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-003/solr.xml
   [junit4]   2> 1227431 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-003/.
   [junit4]   2> 1227431 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.CoreContainer New CoreContainer 1316711039
   [junit4]   2> 1227431 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-003]
   [junit4]   2> 1227431 WARN  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-003/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-003/lib
   [junit4]   2> 1227433 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1227434 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1227434 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1227434 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1227434 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1227434 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1227435 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-003/.
   [junit4]   2> 1227436 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {name=collection1, loadOnStartup=true, transient=false, schema=schema.xml, configSetProperties=configsetprops.json, config=solrconfig.xml, dataDir=data/}
   [junit4]   2> 1227436 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-003/./collection1
   [junit4]   2> 1227436 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 1227437 INFO  (coreLoadExecutor-4470-thread-1) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-003/collection1'
   [junit4]   2> 1227444 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0
   [junit4]   2> 1227444 INFO  (TEST-TestReplicationHandler.doTestRepeater-seed#[DA5873B6F3FA9AF8]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1227444 INFO  (coreLoadExecutor-4470-thread-1) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1227444 INFO  (coreLoadExecutor-4470-thread-1) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1227454 INFO  (coreLoadExecutor-4470-thread-1) [    ] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 1227475 INFO  (coreLoadExecutor-4470-thread-1) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.5.0
   [junit4]   2> 1227517 INFO  (coreLoadExecutor-4470-thread-1) [    ] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1227521 INFO  (coreLoadExecutor-4470-thread-1) [    ] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1227528 INFO  (coreLoadExecutor-4470-thread-1) [    ] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 1227530 INFO  (coreLoadExecutor-4470-thread-1) [    ] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-003/collection1'
   [junit4]   2> 1227530 INFO  (coreLoadExecutor-4470-thread-1) [    ] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-003/./collection1
   [junit4]   2> 1227530 INFO  (coreLoadExecutor-4470-thread-1) [    x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 1227530 INFO  (coreLoadExecutor-4470-thread-1) [    x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-003/collection1], dataDir=[null]
   [junit4]   2> 1227530 INFO  (coreLoadExecutor-4470-thread-1) [    x:collection1] o.a.s.c.SolrCore JMX monitoring not detected for core: collection1
   [junit4]   2> 1227531 INFO  (coreLoadExecutor-4470-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-003/./collection1/data
   [junit4]   2> 1227531 INFO  (coreLoadExecutor-4470-thread-1) [    x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-003/./collection1/data/index/
   [junit4]   2> 1227532 WARN  (coreLoadExecutor-4470-thread-1) [    x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-003/./collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 1227532 INFO  (coreLoadExecutor-4470-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-003/./collection1/data/index
   [junit4]   2> 1227532 INFO  (coreLoadExecutor-4470-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=19, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.36844622650321657]
   [junit4]   2> 1227532 INFO  (coreLoadExecutor-4470-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=RawDirectoryWrapper(RAMDirectory@191f41b8 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5313a3ea),segFN=segments_1,generation=1}
   [junit4]   2> 1227532 INFO  (coreLoadExecutor-4470-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 1227534 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [    x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-003/./collection1/data/
   [junit4]   2> 1227536 INFO  (coreLoadExecutor-4470-thread-1) [    x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 1227538 INFO  (coreLoadExecutor-4470-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1227539 INFO  (coreLoadExecutor-4470-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1227540 INFO  (coreLoadExecutor-4470-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1227541 INFO  (coreLoadExecutor-4470-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1227542 INFO  (coreLoadExecutor-4470-thread-1) [    x:collection1] o.a.s.c.PluginBag requestHandler : 'lazy' created with startup=lazy 
   [junit4]   2> 1227543 INFO  (coreLoadExecutor-4470-thread-1) [    x:collection1] o.a.s.c.RequestHandlers Registered paths: lazy,/admin/ping,/admin/properties,/update,/admin/luke,/admin/system,/admin/mbeans,/config,/update/json/docs,/admin/threads,/admin/plugins,/replication,/schema,/get,/update/json,standard,/admin/file,/update/csv,defaults,/admin/logging,/admin/segments
   [junit4]   2> 1227544 INFO  (coreLoadExecutor-4470-thread-1) [    x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 1227544 INFO  (coreLoadExecutor-4470-thread-1) [    x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1227544 INFO  (coreLoadExecutor-4470-thread-1) [    x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1227544 INFO  (coreLoadExecutor-4470-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@6f0d3507
   [junit4]   2> 1227545 INFO  (coreLoadExecutor-4470-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=RawDirectoryWrapper(RAMDirectory@191f41b8 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5313a3ea),segFN=segments_1,generation=1}
   [junit4]   2> 1227545 INFO  (coreLoadExecutor-4470-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 1227545 INFO  (coreLoadExecutor-4470-thread-1) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@5be8056f[collection1] main
   [junit4]   2> 1227545 INFO  (coreLoadExecutor-4470-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-003/collection1/conf
   [junit4]   2> 1227545 INFO  (coreLoadExecutor-4470-thread-1) [    x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {storageDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-003/collection1/conf}
   [junit4]   2> 1227545 INFO  (coreLoadExecutor-4470-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using file:dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-003/collection1/conf
   [junit4]   2> 1227545 INFO  (coreLoadExecutor-4470-thread-1) [    x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 1227546 WARN  (coreLoadExecutor-4470-thread-1) [    x:collection1] o.a.s.h.IndexFetcher 'masterUrl' must be specified without the /replication suffix
   [junit4]   2> 1227547 INFO  (coreLoadExecutor-4470-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler  No value set for 'pollInterval'. Timer Task not started.
   [junit4]   2> 1227547 INFO  (coreLoadExecutor-4470-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler Replication enabled for following config files: schema.xml
   [junit4]   2> 1227547 INFO  (coreLoadExecutor-4470-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1227547 INFO  (coreLoadExecutor-4470-thread-1) [    x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 1227547 INFO  (searcherExecutor-4471-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@5be8056f[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1227950 INFO  (qtp755704053-11237) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[0]} 0 0
   [junit4]   2> 1227953 INFO  (qtp755704053-11236) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[1]} 0 0
   [junit4]   2> 1227955 INFO  (qtp755704053-11238) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[2]} 0 0
   [junit4]   2> 1227957 INFO  (qtp755704053-11239) [    x:collection1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 1227959 INFO  (qtp755704053-11239) [    x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 	commit{dir=RawDirectoryWrapper(RAMDirectory@569f4e3e lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@74c9f4df),segFN=segments_1,generation=1}
   [junit4]   2> 	commit{dir=RawDirectoryWrapper(RAMDirectory@569f4e3e lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@74c9f4df),segFN=segments_2,generation=2}
   [junit4]   2> 1227960 INFO  (qtp755704053-11239) [    x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 2
   [junit4]   2> 1227962 INFO  (qtp755704053-11239) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@2556203e[collection1] main
   [junit4]   2> 1227962 INFO  (qtp755704053-11239) [    x:collection1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 1227962 INFO  (searcherExecutor-4440-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@2556203e[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(5.5.0):C3)))}
   [junit4]   2> 1227962 INFO  (qtp755704053-11239) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={waitSearcher=true&softCommit=false&commit=true&wt=javabin&version=2} {commit=} 0 4
   [junit4]   2> 1227966 WARN  (explicit-fetchindex-cmd) [    x:collection1] o.a.s.h.IndexFetcher 'masterUrl' must be specified without the /replication suffix
   [junit4]   2> 1227972 INFO  (qtp755704053-11240) [    x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=0 
   [junit4]   2> 1227972 INFO  (explicit-fetchindex-cmd) [    x:collection1] o.a.s.h.IndexFetcher Master's generation: 2
   [junit4]   2> 1227972 INFO  (explicit-fetchindex-cmd) [    x:collection1] o.a.s.h.IndexFetcher Slave's generation: 1
   [junit4]   2> 1227972 INFO  (explicit-fetchindex-cmd) [    x:collection1] o.a.s.h.IndexFetcher Starting replication process
   [junit4]   2> 1227977 INFO  (qtp755704053-11237) [    x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={command=filelist&qt=/replication&generation=2&wt=javabin&version=2} status=0 QTime=2 
   [junit4]   2> 1227977 INFO  (explicit-fetchindex-cmd) [    x:collection1] o.a.s.h.IndexFetcher Number of files in latest index in master: 10
   [junit4]   2> 1227980 INFO  (explicit-fetchindex-cmd) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/solr-instance-003/./collection1/data/index.20151207024828861
   [junit4]   2> 1227980 INFO  (explicit-fetchindex-cmd) [    x:collection1] o.a.s.u.DefaultSolrCoreState Rollback old IndexWriter... core=collection1
   [junit4]   2> 1227982 INFO  (explicit-fetchindex-cmd) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=18, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
   [junit4]   2> 1227982 INFO  (explicit-fetchindex-cmd) [    x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=RawDirectoryWrapper(RAMDirectory@191f41b8 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5313a3ea),segFN=segments_1,generation=1}
   [junit4]   2> 1227982 INFO  (explicit-fetchindex-cmd) [    x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 1227982 INFO  (explicit-fetchindex-cmd) [    x:collection1] o.a.s.u.DefaultSolrCoreState New IndexWriter is ready to be used.
   [junit4]   2> 1227982 INFO  (explicit-fetchindex-cmd) [    x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=RawDirectoryWrapper(RAMDirectory@191f41b8 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5313a3ea),segFN=segments_1,generation=1}
   [junit4]   2> 1227982 INFO  (explicit-fetchindex-cmd) [    x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 1227982 INFO  (explicit-fetchindex-cmd) [    x:collection1] o.a.s.u.DefaultSolrCoreState Rollback old IndexWriter... core=collection1
   [junit4]   2> 1227982 INFO  (explicit-fetchindex-cmd) [    x:collection1] o.a.s.h.IndexFetcher Starting download (fullCopy=false) to MockDirectoryWrapper(NRTCachingDirectory(NIOFSDirectory@/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DA5873B6F3FA9AF8-001/index-NIOFSDirectory-005 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7a3dbf04; maxCacheMB=0.0126953125 maxMergeSizeMB=0.09375))
   [junit4]   2> 1227995 INFO  (qtp755704053-11236) [    x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={checksum=true&file=_0.fdx&command=filecontent&qt=/replication&generation=2&wt=filestream} status=0 QTime=3 
   [junit4]   2> 1228000 INFO  (qtp755704053-11238) [    x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={checksum=true&file=_0.nvd&command=filecontent&qt=/replication&generation=2&wt=filestream} status=0 QTime=0 
   [junit4]   2> 1228004 INFO  (qtp755704053-11239) [    x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={checksum=true&file=_0_Lucene50_0.doc&command=filecontent&qt=/replication&generation=2&wt=filestream} status=0 QTime=0 
   [junit4]   2> 1228007 INFO  (qtp755704053

[...truncated too long message...]

/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-admin-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-authn-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-authz-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-changelog-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-collective-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-event-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-exception-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-journal-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-normalization-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-operational-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-referral-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-schema-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-subtree-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-trigger-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-jdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-jdbm1-2.0.0-M2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-kerberos-codec-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-ldif-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-mavibot-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-protocol-kerberos-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-protocol-ldap-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-protocol-shared-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-xdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/api-all-1.0.0-M20.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/bcprov-jdk15-1.45.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/cglib-nodep-2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/commons-collections-3.2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/commons-math3-3.4.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/easymock-3.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/ehcache-core-2.4.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/hadoop-common-2.6.0-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/hadoop-hdfs-2.6.0-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/hadoop-minikdc-2.6.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jackson-annotations-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jackson-databind-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jersey-core-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jersey-server-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jetty-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jetty-util-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/mina-core-2.0.0-M5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/objenesis-1.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/analysis/icu/lucene-analyzers-icu-5.5.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-analysis-extras/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/contrib/analysis-extras/lib/icu4j-54.1.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-launcher.jar:/x1/jenkins/.ant/lib/ivy-2.3.0.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-antlr.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-regexp.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-resolver.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-net.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jmf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-oro.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-xalan2.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-logging.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jsch.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit4.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-javamail.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jai.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jdepend.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bsf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-swing.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-netrexx.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bcel.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-testutil.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-log4j.jar:/x1/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.7/lib/tools.jar:/x1/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.3.1.jar com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J2-20151206_152757_655.events @/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J2-20151206_152757_655.suites -stdin
   [junit4] ERROR: JVM J2 ended with an exception: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1503)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:130)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:963)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:960)
   [junit4] 	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4] 	at java.lang.Thread.run(Thread.java:745)

BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:799: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:736: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: At least one slave process threw an exception, first: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.

Total time: 221 minutes 43 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1037 - Still Failing

[Apache Jenkins Server <je...@builds.apache.org>]

Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1037/

2 tests failed.
FAILED:  org.apache.lucene.search.TestGeoPointQuery.testRandomBig

Error Message:
Test abandoned because suite timeout was reached.

Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
	at __randomizedtesting.SeedInfo.seed([9F451249890883AF]:0)


FAILED:  junit.framework.TestSuite.org.apache.lucene.search.TestGeoPointQuery

Error Message:
Suite timeout exceeded (>= 7200000 msec).

Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
	at __randomizedtesting.SeedInfo.seed([9F451249890883AF]:0)




Build Log:
[...truncated 8703 lines...]
   [junit4] Suite: org.apache.lucene.search.TestGeoPointQuery
   [junit4]   2> ??? 05, 2015 1:58:23 ?? com.carrotsearch.randomizedtesting.ThreadLeakControl$2 evaluate
   [junit4]   2> WARNING: Suite execution timed out: org.apache.lucene.search.TestGeoPointQuery
   [junit4]   2>    1) Thread[id=1, name=main, state=WAITING, group=main]
   [junit4]   2>         at java.lang.Object.wait(Native Method)
   [junit4]   2>         at java.lang.Thread.join(Thread.java:1281)
   [junit4]   2>         at java.lang.Thread.join(Thread.java:1355)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:601)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner.run(RandomizedRunner.java:450)
   [junit4]   2>         at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.execute(SlaveMain.java:244)
   [junit4]   2>         at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.main(SlaveMain.java:355)
   [junit4]   2>         at com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe.main(SlaveMainSafe.java:10)
   [junit4]   2>    2) Thread[id=38, name=T2, state=RUNNABLE, group=TGRP-TestGeoPointQuery]
   [junit4]   2>         at java.lang.Long.valueOf(Long.java:577)
   [junit4]   2>         at org.apache.lucene.util.GeoUtils.mortonHash(GeoUtils.java:50)
   [junit4]   2>         at org.apache.lucene.search.TestGeoPointQuery.radiusQueryCanBeWrong(TestGeoPointQuery.java:206)
   [junit4]   2>         at org.apache.lucene.search.TestGeoPointQuery.distanceRangeContainsPoint(TestGeoPointQuery.java:192)
   [junit4]   2>         at org.apache.lucene.util.BaseGeoPointTestCase$2$2.shouldMatch(BaseGeoPointTestCase.java:701)
   [junit4]   2>         at org.apache.lucene.util.BaseGeoPointTestCase$VerifyHits.test(BaseGeoPointTestCase.java:526)
   [junit4]   2>         at org.apache.lucene.util.BaseGeoPointTestCase$2._run(BaseGeoPointTestCase.java:753)
   [junit4]   2>         at org.apache.lucene.util.BaseGeoPointTestCase$2.run(BaseGeoPointTestCase.java:618)
   [junit4]   2>    3) Thread[id=8, name=JUnit4-serializer-daemon, state=TIMED_WAITING, group=main]
   [junit4]   2>         at java.lang.Thread.sleep(Native Method)
   [junit4]   2>         at com.carrotsearch.ant.tasks.junit4.events.Serializer$1.run(Serializer.java:47)
   [junit4]   2>    4) Thread[id=16, name=SUITE-TestGeoPointQuery-seed#[9F451249890883AF], state=RUNNABLE, group=TGRP-TestGeoPointQuery]
   [junit4]   2>         at java.lang.Thread.getStackTrace(Thread.java:1589)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$4.run(ThreadLeakControl.java:688)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$4.run(ThreadLeakControl.java:685)
   [junit4]   2>         at java.security.AccessController.doPrivileged(Native Method)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl.getStackTrace(ThreadLeakControl.java:685)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl.getThreadsWithTraces(ThreadLeakControl.java:701)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl.formatThreadStacksFull(ThreadLeakControl.java:681)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl.access$1000(ThreadLeakControl.java:64)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$2.evaluate(ThreadLeakControl.java:414)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:681)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner.access$200(RandomizedRunner.java:140)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$2.run(RandomizedRunner.java:591)
   [junit4]   2>    5) Thread[id=17, name=TEST-TestGeoPointQuery.testRandomBig-seed#[9F451249890883AF], state=WAITING, group=TGRP-TestGeoPointQuery]
   [junit4]   2>         at java.lang.Object.wait(Native Method)
   [junit4]   2>         at java.lang.Thread.join(Thread.java:1281)
   [junit4]   2>         at java.lang.Thread.join(Thread.java:1355)
   [junit4]   2>         at org.apache.lucene.util.BaseGeoPointTestCase.verify(BaseGeoPointTestCase.java:764)
   [junit4]   2>         at org.apache.lucene.util.BaseGeoPointTestCase.doTestRandom(BaseGeoPointTestCase.java:399)
   [junit4]   2>         at org.apache.lucene.util.BaseGeoPointTestCase.testRandomBig(BaseGeoPointTestCase.java:327)
   [junit4]   2>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   2>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   [junit4]   2>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2>         at java.lang.reflect.Method.invoke(Method.java:606)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
   [junit4]   2>         at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
   [junit4]   2>         at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2>         at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
   [junit4]   2>         at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2>         at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
   [junit4]   2>         at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   2>         at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2>         at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2>         at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
   [junit4]   2>         at java.lang.Thread.run(Thread.java:745)
   [junit4]   2>    6) Thread[id=36, name=T0, state=RUNNABLE, group=TGRP-TestGeoPointQuery]
   [junit4]   2>         at org.apache.lucene.util.GeoUtils.rectCrossesPoly(GeoUtils.java:208)
   [junit4]   2>         at org.apache.lucene.util.GeoUtils.rectWithinPoly(GeoUtils.java:329)
   [junit4]   2>         at org.apache.lucene.search.GeoPointInPolygonQuery$GeoPolygonTermsEnum.cellWithin(GeoPointInPolygonQuery.java:156)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:104)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.<init>(GeoPointTermsEnum.java:64)
   [junit4]   2>         at org.apache.lucene.search.GeoPointInPolygonQuery$GeoPolygonTermsEnum.<init>(GeoPointInPolygonQuery.java:145)
   [junit4]   2>         at org.apache.lucene.search.GeoPointInPolygonQuery.getTermsEnum(GeoPointInPolygonQuery.java:84)
   [junit4]   2>         at org.apache.lucene.search.MultiTermQuery.getTermsEnum(MultiTermQuery.java:318)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermQueryConstantScoreWrapper$1.getDocIDs(GeoPointTermQueryConstantScoreWrapper.java:72)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermQueryConstantScoreWrapper$1.bulkScorer(GeoPointTermQueryConstantScoreWrapper.java:117)
   [junit4]   2>         at org.apache.lucene.search.LRUQueryCache$CachingWrapperWeight.bulkScorer(LRUQueryCache.java:655)
   [junit4]   2>         at org.apache.lucene.search.AssertingWeight.bulkScorer(AssertingWeight.java:69)
   [junit4]   2>         at org.apache.lucene.search.AssertingWeight.bulkScorer(AssertingWeight.java:69)
   [junit4]   2>         at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:818)
   [junit4]   2>         at org.apache.lucene.search.AssertingIndexSearcher.search(AssertingIndexSearcher.java:92)
   [junit4]   2>         at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:535)
   [junit4]   2>         at org.apache.lucene.util.BaseGeoPointTestCase$VerifyHits.test(BaseGeoPointTestCase.java:496)
   [junit4]   2>         at org.apache.lucene.util.BaseGeoPointTestCase$2._run(BaseGeoPointTestCase.java:753)
   [junit4]   2>         at org.apache.lucene.util.BaseGeoPointTestCase$2.run(BaseGeoPointTestCase.java:618)
   [junit4]   2>    7) Thread[id=37, name=T1, state=BLOCKED, group=TGRP-TestGeoPointQuery]
   [junit4]   2>         at org.apache.lucene.util.GeoUtils.rectCrossesPoly(GeoUtils.java:208)
   [junit4]   2>         at org.apache.lucene.util.GeoUtils.rectWithinPoly(GeoUtils.java:329)
   [junit4]   2>         at org.apache.lucene.search.GeoPointInPolygonQuery$GeoPolygonTermsEnum.cellWithin(GeoPointInPolygonQuery.java:156)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:104)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.<init>(GeoPointTermsEnum.java:64)
   [junit4]   2>         at org.apache.lucene.search.GeoPointInPolygonQuery$GeoPolygonTermsEnum.<init>(GeoPointInPolygonQuery.java:145)
   [junit4]   2>         at org.apache.lucene.search.GeoPointInPolygonQuery.getTermsEnum(GeoPointInPolygonQuery.java:84)
   [junit4]   2>         at org.apache.lucene.search.MultiTermQuery.getTermsEnum(MultiTermQuery.java:318)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermQueryConstantScoreWrapper$1.getDocIDs(GeoPointTermQueryConstantScoreWrapper.java:72)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermQueryConstantScoreWrapper$1.bulkScorer(GeoPointTermQueryConstantScoreWrapper.java:117)
   [junit4]   2>         at org.apache.lucene.search.LRUQueryCache$CachingWrapperWeight.cache(LRUQueryCache.java:604)
   [junit4]   2>         at org.apache.lucene.search.LRUQueryCache$CachingWrapperWeight.bulkScorer(LRUQueryCache.java:652)
   [junit4]   2>         at org.apache.lucene.search.AssertingWeight.bulkScorer(AssertingWeight.java:69)
   [junit4]   2>         at org.apache.lucene.search.AssertingWeight.bulkScorer(AssertingWeight.java:69)
   [junit4]   2>         at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:818)
   [junit4]   2>         at org.apache.lucene.search.AssertingIndexSearcher.search(AssertingIndexSearcher.java:92)
   [junit4]   2>         at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:535)
   [junit4]   2>         at org.apache.lucene.util.BaseGeoPointTestCase$VerifyHits.test(BaseGeoPointTestCase.java:496)
   [junit4]   2>         at org.apache.lucene.util.BaseGeoPointTestCase$2._run(BaseGeoPointTestCase.java:753)
   [junit4]   2>         at org.apache.lucene.util.BaseGeoPointTestCase$2.run(BaseGeoPointTestCase.java:618)
   [junit4]   2>    8) Thread[id=39, name=T3, state=RUNNABLE, group=TGRP-TestGeoPointQuery]
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:109)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.<init>(GeoPointTermsEnum.java:64)
   [junit4]   2>         at org.apache.lucene.search.GeoPointDistanceQueryImpl$GeoPointRadiusTermsEnum.<init>(GeoPointDistanceQueryImpl.java:58)
   [junit4]   2>         at org.apache.lucene.search.GeoPointDistanceQueryImpl.getTermsEnum(GeoPointDistanceQueryImpl.java:47)
   [junit4]   2>         at org.apache.lucene.search.MultiTermQuery.getTermsEnum(MultiTermQuery.java:318)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermQueryConstantScoreWrapper$1.getDocIDs(GeoPointTermQueryConstantScoreWrapper.java:72)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermQueryConstantScoreWrapper$1.bulkScorer(GeoPointTermQueryConstantScoreWrapper.java:117)
   [junit4]   2>         at org.apache.lucene.search.LRUQueryCache$CachingWrapperWeight.cache(LRUQueryCache.java:604)
   [junit4]   2>         at org.apache.lucene.search.LRUQueryCache$CachingWrapperWeight.bulkScorer(LRUQueryCache.java:652)
   [junit4]   2>         at org.apache.lucene.search.AssertingWeight.bulkScorer(AssertingWeight.java:69)
   [junit4]   2>         at org.apache.lucene.search.BooleanWeight.booleanScorer(BooleanWeight.java:198)
   [junit4]   2>         at org.apache.lucene.search.BooleanWeight.bulkScorer(BooleanWeight.java:239)
   [junit4]   2>         at org.apache.lucene.search.LRUQueryCache$CachingWrapperWeight.bulkScorer(LRUQueryCache.java:655)
   [junit4]   2>         at org.apache.lucene.search.AssertingWeight.bulkScorer(AssertingWeight.java:69)
   [junit4]   2>         at org.apache.lucene.search.AssertingWeight.bulkScorer(AssertingWeight.java:69)
   [junit4]   2>         at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:818)
   [junit4]   2>         at org.apache.lucene.search.AssertingIndexSearcher.search(AssertingIndexSearcher.java:92)
   [junit4]   2>         at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:535)
   [junit4]   2>         at org.apache.lucene.util.BaseGeoPointTestCase$VerifyHits.test(BaseGeoPointTestCase.java:496)
   [junit4]   2>         at org.apache.lucene.util.BaseGeoPointTestCase$2._run(BaseGeoPointTestCase.java:753)
   [junit4]   2>         at org.apache.lucene.util.BaseGeoPointTestCase$2.run(BaseGeoPointTestCase.java:618)
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestGeoPointQuery -Dtests.method=testRandomBig -Dtests.seed=9F451249890883AF -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=el_CY -Dtests.timezone=America/Argentina/Jujuy -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
   [junit4] ERROR   7416s J1 | TestGeoPointQuery.testRandomBig <<<
   [junit4]    > Throwable #1: java.lang.Exception: Test abandoned because suite timeout was reached.
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([9F451249890883AF]:0)
   [junit4]   2> ??? 05, 2015 2:18:43 ?? com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 5 leaked thread(s).
   [junit4]   2> ??? 05, 2015 2:28:02 ?? com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> SEVERE: 5 threads leaked from SUITE scope at org.apache.lucene.search.TestGeoPointQuery: 
   [junit4]   2>    1) Thread[id=17, name=TEST-TestGeoPointQuery.testRandomBig-seed#[9F451249890883AF], state=WAITING, group=TGRP-TestGeoPointQuery]
   [junit4]   2>         at java.lang.Object.wait(Native Method)
   [junit4]   2>         at java.lang.Thread.join(Thread.java:1281)
   [junit4]   2>         at java.lang.Thread.join(Thread.java:1355)
   [junit4]   2>         at org.apache.lucene.util.BaseGeoPointTestCase.verify(BaseGeoPointTestCase.java:764)
   [junit4]   2>         at org.apache.lucene.util.BaseGeoPointTestCase.doTestRandom(BaseGeoPointTestCase.java:399)
   [junit4]   2>         at org.apache.lucene.util.BaseGeoPointTestCase.testRandomBig(BaseGeoPointTestCase.java:327)
   [junit4]   2>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   2>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   [junit4]   2>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2>         at java.lang.reflect.Method.invoke(Method.java:606)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
   [junit4]   2>         at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
   [junit4]   2>         at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2>         at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
   [junit4]   2>         at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2>         at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
   [junit4]   2>         at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   2>         at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2>         at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2>         at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
   [junit4]   2>         at java.lang.Thread.run(Thread.java:745)
   [junit4]   2>    2) Thread[id=36, name=T0, state=RUNNABLE, group=TGRP-TestGeoPointQuery]
   [junit4]   2>         at org.apache.lucene.util.GeoUtils.rectCrossesPoly(GeoUtils.java:208)
   [junit4]   2>         at org.apache.lucene.util.GeoUtils.rectWithinPoly(GeoUtils.java:329)
   [junit4]   2>         at org.apache.lucene.search.GeoPointInPolygonQuery$GeoPolygonTermsEnum.cellWithin(GeoPointInPolygonQuery.java:156)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:104)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.<init>(GeoPointTermsEnum.java:64)
   [junit4]   2>         at org.apache.lucene.search.GeoPointInPolygonQuery$GeoPolygonTermsEnum.<init>(GeoPointInPolygonQuery.java:145)
   [junit4]   2>         at org.apache.lucene.search.GeoPointInPolygonQuery.getTermsEnum(GeoPointInPolygonQuery.java:84)
   [junit4]   2>         at org.apache.lucene.search.MultiTermQuery.getTermsEnum(MultiTermQuery.java:318)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermQueryConstantScoreWrapper$1.getDocIDs(GeoPointTermQueryConstantScoreWrapper.java:72)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermQueryConstantScoreWrapper$1.bulkScorer(GeoPointTermQueryConstantScoreWrapper.java:117)
   [junit4]   2>         at org.apache.lucene.search.LRUQueryCache$CachingWrapperWeight.bulkScorer(LRUQueryCache.java:655)
   [junit4]   2>         at org.apache.lucene.search.AssertingWeight.bulkScorer(AssertingWeight.java:69)
   [junit4]   2>         at org.apache.lucene.search.AssertingWeight.bulkScorer(AssertingWeight.java:69)
   [junit4]   2>         at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:818)
   [junit4]   2>         at org.apache.lucene.search.AssertingIndexSearcher.search(AssertingIndexSearcher.java:92)
   [junit4]   2>         at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:535)
   [junit4]   2>         at org.apache.lucene.util.BaseGeoPointTestCase$VerifyHits.test(BaseGeoPointTestCase.java:496)
   [junit4]   2>         at org.apache.lucene.util.BaseGeoPointTestCase$2._run(BaseGeoPointTestCase.java:753)
   [junit4]   2>         at org.apache.lucene.util.BaseGeoPointTestCase$2.run(BaseGeoPointTestCase.java:618)
   [junit4]   2>    3) Thread[id=37, name=T1, state=RUNNABLE, group=TGRP-TestGeoPointQuery]
   [junit4]   2>         at org.apache.lucene.util.GeoUtils.rectCrossesPoly(GeoUtils.java:208)
   [junit4]   2>         at org.apache.lucene.util.GeoUtils.rectWithinPoly(GeoUtils.java:329)
   [junit4]   2>         at org.apache.lucene.search.GeoPointInPolygonQuery$GeoPolygonTermsEnum.cellWithin(GeoPointInPolygonQuery.java:156)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:104)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.<init>(GeoPointTermsEnum.java:64)
   [junit4]   2>         at org.apache.lucene.search.GeoPointInPolygonQuery$GeoPolygonTermsEnum.<init>(GeoPointInPolygonQuery.java:145)
   [junit4]   2>         at org.apache.lucene.search.GeoPointInPolygonQuery.getTermsEnum(GeoPointInPolygonQuery.java:84)
   [junit4]   2>         at org.apache.lucene.search.MultiTermQuery.getTermsEnum(MultiTermQuery.java:318)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermQueryConstantScoreWrapper$1.getDocIDs(GeoPointTermQueryConstantScoreWrapper.java:72)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermQueryConstantScoreWrapper$1.bulkScorer(GeoPointTermQueryConstantScoreWrapper.java:117)
   [junit4]   2>         at org.apache.lucene.search.LRUQueryCache$CachingWrapperWeight.cache(LRUQueryCache.java:604)
   [junit4]   2>         at org.apache.lucene.search.LRUQueryCache$CachingWrapperWeight.bulkScorer(LRUQueryCache.java:652)
   [junit4]   2>         at org.apache.lucene.search.AssertingWeight.bulkScorer(AssertingWeight.java:69)
   [junit4]   2>         at org.apache.lucene.search.AssertingWeight.bulkScorer(AssertingWeight.java:69)
   [junit4]   2>         at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:818)
   [junit4]   2>         at org.apache.lucene.search.AssertingIndexSearcher.search(AssertingIndexSearcher.java:92)
   [junit4]   2>         at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:535)
   [junit4]   2>         at org.apache.lucene.util.BaseGeoPointTestCase$VerifyHits.test(BaseGeoPointTestCase.java:496)
   [junit4]   2>         at org.apache.lucene.util.BaseGeoPointTestCase$2._run(BaseGeoPointTestCase.java:753)
   [junit4]   2>         at org.apache.lucene.util.BaseGeoPointTestCase$2.run(BaseGeoPointTestCase.java:618)
   [junit4]   2>    4) Thread[id=39, name=T3, state=RUNNABLE, group=TGRP-TestGeoPointQuery]
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:109)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.<init>(GeoPointTermsEnum.java:64)
   [junit4]   2>         at org.apache.lucene.search.GeoPointDistanceQueryImpl$GeoPointRadiusTermsEnum.<init>(GeoPointDistanceQueryImpl.java:58)
   [junit4]   2>         at org.apache.lucene.search.GeoPointDistanceQueryImpl.getTermsEnum(GeoPointDistanceQueryImpl.java:47)
   [junit4]   2>         at org.apache.lucene.search.MultiTermQuery.getTermsEnum(MultiTermQuery.java:318)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermQueryConstantScoreWrapper$1.getDocIDs(GeoPointTermQueryConstantScoreWrapper.java:72)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermQueryConstantScoreWrapper$1.bulkScorer(GeoPointTermQueryConstantScoreWrapper.java:117)
   [junit4]   2>         at org.apache.lucene.search.LRUQueryCache$CachingWrapperWeight.bulkScorer(LRUQueryCache.java:655)
   [junit4]   2>         at org.apache.lucene.search.AssertingWeight.bulkScorer(AssertingWeight.java:69)
   [junit4]   2>         at org.apache.lucene.search.BooleanWeight.booleanScorer(BooleanWeight.java:198)
   [junit4]   2>         at org.apache.lucene.search.BooleanWeight.bulkScorer(BooleanWeight.java:239)
   [junit4]   2>         at org.apache.lucene.search.LRUQueryCache$CachingWrapperWeight.bulkScorer(LRUQueryCache.java:655)
   [junit4]   2>         at org.apache.lucene.search.AssertingWeight.bulkScorer(AssertingWeight.java:69)
   [junit4]   2>         at org.apache.lucene.search.AssertingWeight.bulkScorer(AssertingWeight.java:69)
   [junit4]   2>         at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:818)
   [junit4]   2>         at org.apache.lucene.search.AssertingIndexSearcher.search(AssertingIndexSearcher.java:92)
   [junit4]   2>         at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:535)
   [junit4]   2>         at org.apache.lucene.util.BaseGeoPointTestCase$VerifyHits.test(BaseGeoPointTestCase.java:496)
   [junit4]   2>         at org.apache.lucene.util.BaseGeoPointTestCase$2._run(BaseGeoPointTestCase.java:753)
   [junit4]   2>         at org.apache.lucene.util.BaseGeoPointTestCase$2.run(BaseGeoPointTestCase.java:618)
   [junit4]   2>    5) Thread[id=38, name=T2, state=BLOCKED, group=TGRP-TestGeoPointQuery]
   [junit4]   2>         at java.lang.Long.valueOf(Long.java:577)
   [junit4]   2>         at org.apache.lucene.util.GeoUtils.mortonHash(GeoUtils.java:50)
   [junit4]   2>         at org.apache.lucene.search.TestGeoPointQuery.radiusQueryCanBeWrong(TestGeoPointQuery.java:203)
   [junit4]   2>         at org.apache.lucene.search.TestGeoPointQuery.distanceRangeContainsPoint(TestGeoPointQuery.java:192)
   [junit4]   2>         at org.apache.lucene.util.BaseGeoPointTestCase$2$2.shouldMatch(BaseGeoPointTestCase.java:701)
   [junit4]   2>         at org.apache.lucene.util.BaseGeoPointTestCase$VerifyHits.test(BaseGeoPointTestCase.java:526)
   [junit4]   2>         at org.apache.lucene.util.BaseGeoPointTestCase$2._run(BaseGeoPointTestCase.java:753)
   [junit4]   2>         at org.apache.lucene.util.BaseGeoPointTestCase$2.run(BaseGeoPointTestCase.java:618)
   [junit4]   2> ??? 05, 2015 2:30:42 ?? com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
   [junit4]   2> INFO: Starting to interrupt leaked threads:
   [junit4]   2>    1) Thread[id=17, name=TEST-TestGeoPointQuery.testRandomBig-seed#[9F451249890883AF], state=WAITING, group=TGRP-TestGeoPointQuery]
   [junit4]   2>    2) Thread[id=36, name=T0, state=RUNNABLE, group=TGRP-TestGeoPointQuery]
   [junit4]   2>    3) Thread[id=37, name=T1, state=BLOCKED, group=TGRP-TestGeoPointQuery]
   [junit4]   2>    4) Thread[id=39, name=T3, state=RUNNABLE, group=TGRP-TestGeoPointQuery]
   [junit4]   2>    5) Thread[id=38, name=T2, state=BLOCKED, group=TGRP-TestGeoPointQuery]
   [junit4]   2> ??? 05, 2015 2:36:12 ?? com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
   [junit4]   2> SEVERE: There are still zombie threads that couldn't be terminated:
   [junit4]   2>    1) Thread[id=38, name=T2, state=RUNNABLE, group=TGRP-TestGeoPointQuery]
   [junit4]   2>         at java.lang.Integer.valueOf(Integer.java:642)
   [junit4]   2>         at org.apache.lucene.util.BaseGeoPointTestCase$VerifyHits.test(BaseGeoPointTestCase.java:521)
   [junit4]   2>         at org.apache.lucene.util.BaseGeoPointTestCase$2._run(BaseGeoPointTestCase.java:753)
   [junit4]   2>         at org.apache.lucene.util.BaseGeoPointTestCase$2.run(BaseGeoPointTestCase.java:618)
   [junit4]   2>    2) Thread[id=17, name=SUITE-TestGeoPointQuery-seed#[9F451249890883AF]-worker, state=BLOCKED, group=TGRP-TestGeoPointQuery]
   [junit4]   2>         at java.util.Collections$UnmodifiableMap$UnmodifiableEntrySet$1.next(Collections.java:1403)
   [junit4]   2>         at java.util.Collections$UnmodifiableMap$UnmodifiableEntrySet$1.next(Collections.java:1396)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ClassModel.getAnnotatedLeafMethods(ClassModel.java:254)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner.getShuffledMethods(RandomizedRunner.java:1254)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner.access$500(RandomizedRunner.java:140)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:832)
   [junit4]   2>         at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   2>         at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2>         at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2>         at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
   [junit4]   2>         at java.lang.Thread.run(Thread.java:745)
   [junit4]   2>    3) Thread[id=36, name=T0, state=BLOCKED, group=TGRP-TestGeoPointQuery]
   [junit4]   2>         at org.apache.lucene.util.GeoUtils.rectCrossesPoly(GeoUtils.java:208)
   [junit4]   2>         at org.apache.lucene.search.GeoPointInPolygonQuery$GeoPolygonTermsEnum.cellCrosses(GeoPointInPolygonQuery.java:150)
   [junit4]   2>         at org.apache.lucene.search.GeoPointInPolygonQuery$GeoPolygonTermsEnum.cellIntersectsShape(GeoPointInPolygonQuery.java:162)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:105)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.<init>(GeoPointTermsEnum.java:64)
   [junit4]   2>         at org.apache.lucene.search.GeoPointInPolygonQuery$GeoPolygonTermsEnum.<init>(GeoPointInPolygonQuery.java:145)
   [junit4]   2>         at org.apache.lucene.search.GeoPointInPolygonQuery.getTermsEnum(GeoPointInPolygonQuery.java:84)
   [junit4]   2>         at org.apache.lucene.search.MultiTermQuery.getTermsEnum(MultiTermQuery.java:318)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermQueryConstantScoreWrapper$1.getDocIDs(GeoPointTermQueryConstantScoreWrapper.java:72)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermQueryConstantScoreWrapper$1.bulkScorer(GeoPointTermQueryConstantScoreWrapper.java:117)
   [junit4]   2>         at org.apache.lucene.search.LRUQueryCache$CachingWrapperWeight.bulkScorer(LRUQueryCache.java:655)
   [junit4]   2>         at org.apache.lucene.search.AssertingWeight.bulkScorer(AssertingWeight.java:69)
   [junit4]   2>         at org.apache.lucene.search.AssertingWeight.bulkScorer(AssertingWeight.java:69)
   [junit4]   2>         at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:818)
   [junit4]   2>         at org.apache.lucene.search.AssertingIndexSearcher.search(AssertingIndexSearcher.java:92)
   [junit4]   2>         at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:535)
   [junit4]   2>         at org.apache.lucene.util.BaseGeoPointTestCase$VerifyHits.test(BaseGeoPointTestCase.java:496)
   [junit4]   2>         at org.apache.lucene.util.BaseGeoPointTestCase$2._run(BaseGeoPointTestCase.java:753)
   [junit4]   2>         at org.apache.lucene.util.BaseGeoPointTestCase$2.run(BaseGeoPointTestCase.java:618)
   [junit4]   2>    4) Thread[id=37, name=T1, state=RUNNABLE, group=TGRP-TestGeoPointQuery]
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:111)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.<init>(GeoPointTermsEnum.java:64)
   [junit4]   2>         at org.apache.lucene.search.GeoPointInPolygonQuery$GeoPolygonTermsEnum.<init>(GeoPointInPolygonQuery.java:145)
   [junit4]   2>         at org.apache.lucene.search.GeoPointInPolygonQuery.getTermsEnum(GeoPointInPolygonQuery.java:84)
   [junit4]   2>         at org.apache.lucene.search.MultiTermQuery.getTermsEnum(MultiTermQuery.java:318)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermQueryConstantScoreWrapper$1.getDocIDs(GeoPointTermQueryConstantScoreWrapper.java:72)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermQueryConstantScoreWrapper$1.bulkScorer(GeoPointTermQueryConstantScoreWrapper.java:117)
   [junit4]   2>         at org.apache.lucene.search.LRUQueryCache$CachingWrapperWeight.cache(LRUQueryCache.java:604)
   [junit4]   2>         at org.apache.lucene.search.LRUQueryCache$CachingWrapperWeight.bulkScorer(LRUQueryCache.java:652)
   [junit4]   2>         at org.apache.lucene.search.AssertingWeight.bulkScorer(AssertingWeight.java:69)
   [junit4]   2>         at org.apache.lucene.search.AssertingWeight.bulkScorer(AssertingWeight.java:69)
   [junit4]   2>         at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:818)
   [junit4]   2>         at org.apache.lucene.search.AssertingIndexSearcher.search(AssertingIndexSearcher.java:92)
   [junit4]   2>         at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:535)
   [junit4]   2>         at org.apache.lucene.util.BaseGeoPointTestCase$VerifyHits.test(BaseGeoPointTestCase.java:496)
   [junit4]   2>         at org.apache.lucene.util.BaseGeoPointTestCase$2._run(BaseGeoPointTestCase.java:753)
   [junit4]   2>         at org.apache.lucene.util.BaseGeoPointTestCase$2.run(BaseGeoPointTestCase.java:618)
   [junit4]   2>    5) Thread[id=39, name=T3, state=RUNNABLE, group=TGRP-TestGeoPointQuery]
   [junit4]   2>         at org.apache.lucene.util.GeoUtils.isClosestPointOnRectWithinRange(GeoUtils.java:368)
   [junit4]   2>         at org.apache.lucene.util.GeoUtils.rectCrossesCircle(GeoUtils.java:362)
   [junit4]   2>         at org.apache.lucene.search.GeoPointDistanceQueryImpl$GeoPointRadiusTermsEnum.cellCrosses(GeoPointDistanceQueryImpl.java:80)
   [junit4]   2>         at org.apache.lucene.search.GeoPointDistanceQueryImpl$GeoPointRadiusTermsEnum.cellIntersectsShape(GeoPointDistanceQueryImpl.java:90)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:105)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:84)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.relateAndRecurse(GeoPointTermsEnum.java:114)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.computeRange(GeoPointTermsEnum.java:83)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermsEnum.<init>(GeoPointTermsEnum.java:64)
   [junit4]   2>         at org.apache.lucene.search.GeoPointDistanceQueryImpl$GeoPointRadiusTermsEnum.<init>(GeoPointDistanceQueryImpl.java:58)
   [junit4]   2>         at org.apache.lucene.search.GeoPointDistanceQueryImpl.getTermsEnum(GeoPointDistanceQueryImpl.java:47)
   [junit4]   2>         at org.apache.lucene.search.MultiTermQuery.getTermsEnum(MultiTermQuery.java:318)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermQueryConstantScoreWrapper$1.getDocIDs(GeoPointTermQueryConstantScoreWrapper.java:72)
   [junit4]   2>         at org.apache.lucene.search.GeoPointTermQueryConstantScoreWrapper$1.bulkScorer(GeoPointTermQueryConstantScoreWrapper.java:117)
   [junit4]   2>         at org.apache.lucene.search.LRUQueryCache$CachingWrapperWeight.bulkScorer(LRUQueryCache.java:655)
   [junit4]   2>         at org.apache.lucene.search.AssertingWeight.bulkScorer(AssertingWeight.java:69)
   [junit4]   2>         at org.apache.lucene.search.BooleanWeight.booleanScorer(BooleanWeight.java:198)
   [junit4]   2>         at org.apache.lucene.search.BooleanWeight.bulkScorer(BooleanWeight.java:239)
   [junit4]   2>         at org.apache.lucene.search.LRUQueryCache$CachingWrapperWeight.bulkScorer(LRUQueryCache.java:655)
   [junit4]   2>         at org.apache.lucene.search.AssertingWeight.bulkScorer(AssertingWeight.java:69)
   [junit4]   2>         at org.apache.lucene.search.AssertingWeight.bulkScorer(AssertingWeight.java:69)
   [junit4]   2>         at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:818)
   [junit4]   2>         at org.apache.lucene.search.AssertingIndexSearcher.search(AssertingIndexSearcher.java:92)
   [junit4]   2>         at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:535)
   [junit4]   2>         at org.apache.lucene.util.BaseGeoPointTestCase$VerifyHits.test(BaseGeoPointTestCase.java:496)
   [junit4]   2>         at org.apache.lucene.util.BaseGeoPointTestCase$2._run(BaseGeoPointTestCase.java:753)
   [junit4]   2>         at org.apache.lucene.util.BaseGeoPointTestCase$2.run(BaseGeoPointTestCase.java:618)
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene54): {id=FSTOrd50, point=PostingsFormat(name=Direct)}, docValues:{id=DocValuesFormat(name=Memory), point=DocValuesFormat(name=Direct)}, sim=DefaultSimilarity, locale=el_CY, timezone=America/Argentina/Jujuy
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_80 (64-bit)/cpus=4,threads=6,free=38789464,total=463470592
   [junit4]   2> NOTE: All tests run in this JVM: [DuplicateFilterTest, TestGeoUtils, TestGeoPointQuery]
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestGeoPointQuery -Dtests.seed=9F451249890883AF -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=el_CY -Dtests.timezone=America/Argentina/Jujuy -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
   [junit4] ERROR   0.00s J1 | TestGeoPointQuery (suite) <<<
   [junit4]    > Throwable #1: java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([9F451249890883AF]:0)
   [junit4] Completed [20/20] on J1 in 10245.45s, 23 tests, 2 errors <<< FAILURES!

[...truncated 12 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:799: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:736: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build.xml:472: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:2248: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/module-build.xml:58: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 20 suites, 142 tests, 1 suite-level error, 1 error, 1 ignored (1 assumption) [seed: 9F451249890883AF]

Total time: 291 minutes 58 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1036 - Still Failing

[Apache Jenkins Server <je...@builds.apache.org>]

Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1036/

2 tests failed.
FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=32047, name=collection4, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=32047, name=collection4, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:48527/kn_u: Could not find collection : awholynewstresscollection_collection4_1
	at __randomizedtesting.SeedInfo.seed([42AE944646D3DDD0]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1100)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:871)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:807)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:888)


FAILED:  org.apache.solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR

Error Message:
There are still nodes recoverying - waited for 330 seconds

Stack Trace:
java.lang.AssertionError: There are still nodes recoverying - waited for 330 seconds
	at __randomizedtesting.SeedInfo.seed([42AE944646D3DDD0:9C24FA3B7D71BF23]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:175)
	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:136)
	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:131)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.waitForRecoveriesToFinish(AbstractFullDistribZkTestBase.java:831)
	at org.apache.solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR(LeaderInitiatedRecoveryOnShardRestartTest.java:104)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:965)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:940)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 10717 lines...]
   [junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_42AE944646D3DDD0-001/init-core-data-001
   [junit4]   2> 1001602 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[42AE944646D3DDD0]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false)
   [junit4]   2> 1001604 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[42AE944646D3DDD0]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /kn_u/
   [junit4]   2> 1001618 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 1001618 INFO  (Thread-25889) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1001618 INFO  (Thread-25889) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 1001718 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.ZkTestServer start zk server on port:39818
   [junit4]   2> 1001719 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1001720 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1001724 INFO  (zkCallback-1800-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1878258a name:ZooKeeperConnection Watcher:127.0.0.1:39818 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1001724 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1001724 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1001724 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 1001728 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1001732 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1001732 INFO  (zkCallback-1801-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@53db2de9 name:ZooKeeperConnection Watcher:127.0.0.1:39818/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1001732 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1001733 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1001733 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 1001735 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 1001737 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 1001739 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 1001741 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 1001741 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 1001743 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 1001744 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 1001745 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1001746 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1001747 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 1001747 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 1001749 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 1001749 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 1001751 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 1001751 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 1001753 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 1001754 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 1001757 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1001757 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1001758 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1001758 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1001760 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 1001760 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 1001761 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 1001761 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 1002076 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_42AE944646D3DDD0-001/control-001/cores/collection1
   [junit4]   2> 1002077 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 1002081 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@2dff72ad{/kn_u,null,AVAILABLE}
   [junit4]   2> 1002083 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.e.j.s.ServerConnector Started ServerConnector@5dc1d372{HTTP/1.1}{127.0.0.1:48527}
   [junit4]   2> 1002083 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.e.j.s.Server Started @1005020ms
   [junit4]   2> 1002083 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostPort=48527, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_42AE944646D3DDD0-001/tempDir-001/control/data, hostContext=/kn_u, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_42AE944646D3DDD0-001/control-001/cores}
   [junit4]   2> 1002085 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@7b3cb2c6
   [junit4]   2> 1002085 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_42AE944646D3DDD0-001/control-001'
   [junit4]   2> 1002085 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1002085 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1002086 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1002088 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1002088 INFO  (zkCallback-1802-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@70b6f9a8 name:ZooKeeperConnection Watcher:127.0.0.1:39818/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1002088 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1002089 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1002090 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1002090 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_42AE944646D3DDD0-001/control-001/solr.xml
   [junit4]   2> 1002105 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_42AE944646D3DDD0-001/control-001/cores
   [junit4]   2> 1002105 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.CoreContainer New CoreContainer 1194329061
   [junit4]   2> 1002105 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_42AE944646D3DDD0-001/control-001]
   [junit4]   2> 1002106 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_42AE944646D3DDD0-001/control-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_42AE944646D3DDD0-001/control-001/lib
   [junit4]   2> 1002107 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1002112 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1002113 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1002113 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1002113 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:39818/solr
   [junit4]   2> 1002113 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 1002113 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1002115 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1002116 INFO  (zkCallback-1804-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@567d5ff0 name:ZooKeeperConnection Watcher:127.0.0.1:39818 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1002117 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1002117 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1002119 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [n:127.0.0.1:48527_kn_u    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1002120 INFO  (zkCallback-1805-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@57fe7b3b name:ZooKeeperConnection Watcher:127.0.0.1:39818/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1002120 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [n:127.0.0.1:48527_kn_u    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1002122 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [n:127.0.0.1:48527_kn_u    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 1002124 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [n:127.0.0.1:48527_kn_u    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 1002127 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [n:127.0.0.1:48527_kn_u    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 1002129 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [n:127.0.0.1:48527_kn_u    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 1002132 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [n:127.0.0.1:48527_kn_u    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 1002134 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [n:127.0.0.1:48527_kn_u    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 1002135 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [n:127.0.0.1:48527_kn_u    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 1002137 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [n:127.0.0.1:48527_kn_u    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 1002138 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [n:127.0.0.1:48527_kn_u    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 1002139 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [n:127.0.0.1:48527_kn_u    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:48527_kn_u
   [junit4]   2> 1002140 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [n:127.0.0.1:48527_kn_u    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:48527_kn_u
   [junit4]   2> 1002141 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [n:127.0.0.1:48527_kn_u    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 1002143 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [n:127.0.0.1:48527_kn_u    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 1002144 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [n:127.0.0.1:48527_kn_u    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1002145 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [n:127.0.0.1:48527_kn_u    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94978216905998340-127.0.0.1:48527_kn_u-n_0000000000
   [junit4]   2> 1002145 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [n:127.0.0.1:48527_kn_u    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:48527_kn_u
   [junit4]   2> 1002145 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [n:127.0.0.1:48527_kn_u    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 1002146 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [n:127.0.0.1:48527_kn_u    ] o.a.s.c.Overseer Overseer (id=94978216905998340-127.0.0.1:48527_kn_u-n_0000000000) starting
   [junit4]   2> 1002148 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [n:127.0.0.1:48527_kn_u    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 1002155 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [n:127.0.0.1:48527_kn_u    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 1002156 INFO  (OverseerStateUpdate-94978216905998340-127.0.0.1:48527_kn_u-n_0000000000) [n:127.0.0.1:48527_kn_u    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 1002156 INFO  (OverseerStateUpdate-94978216905998340-127.0.0.1:48527_kn_u-n_0000000000) [n:127.0.0.1:48527_kn_u    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1002167 INFO  (OverseerCollectionConfigSetProcessor-94978216905998340-127.0.0.1:48527_kn_u-n_0000000000) [n:127.0.0.1:48527_kn_u    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 1002170 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [n:127.0.0.1:48527_kn_u    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1002199 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [n:127.0.0.1:48527_kn_u    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1002199 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [n:127.0.0.1:48527_kn_u    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1002200 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [n:127.0.0.1:48527_kn_u    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_42AE944646D3DDD0-001/control-001/cores
   [junit4]   2> 1002201 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [n:127.0.0.1:48527_kn_u    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {collection=control_collection, coreNodeName=, schema=schema.xml, shard=, name=collection1, configSetProperties=configsetprops.json, dataDir=data/, loadOnStartup=true, config=solrconfig.xml, transient=false}
   [junit4]   2> 1002201 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [n:127.0.0.1:48527_kn_u    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_42AE944646D3DDD0-001/control-001/cores/collection1
   [junit4]   2> 1002201 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [n:127.0.0.1:48527_kn_u    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 1002202 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 1002202 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1002202 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [n:127.0.0.1:48527_kn_u    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 1002202 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [n:127.0.0.1:48527_kn_u    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1002205 INFO  (OverseerStateUpdate-94978216905998340-127.0.0.1:48527_kn_u-n_0000000000) [n:127.0.0.1:48527_kn_u    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "node_name":"127.0.0.1:48527_kn_u",
   [junit4]   2>   "base_url":"http://127.0.0.1:48527/kn_u",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "shard":null} current state version: 0
   [junit4]   2> 1002205 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 1002206 INFO  (OverseerStateUpdate-94978216905998340-127.0.0.1:48527_kn_u-n_0000000000) [n:127.0.0.1:48527_kn_u    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "node_name":"127.0.0.1:48527_kn_u",
   [junit4]   2>   "base_url":"http://127.0.0.1:48527/kn_u",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "shard":null}
   [junit4]   2> 1002206 INFO  (OverseerStateUpdate-94978216905998340-127.0.0.1:48527_kn_u-n_0000000000) [n:127.0.0.1:48527_kn_u    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 1002206 INFO  (OverseerStateUpdate-94978216905998340-127.0.0.1:48527_kn_u-n_0000000000) [n:127.0.0.1:48527_kn_u    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 1002208 INFO  (zkCallback-1805-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1002209 INFO  (zkCallback-1805-thread-2-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1003206 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1003206 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 1003207 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 1003208 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 1003209 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1003209 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_42AE944646D3DDD0-001/control-001/cores/collection1'
   [junit4]   2> 1003209 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1003209 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1003211 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 1003221 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 1003234 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.5.0
   [junit4]   2> 1003278 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection   x:collection1] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1003287 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1003532 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 1003540 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 1003542 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 1003553 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1003556 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1003558 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1003559 WARN  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 1003559 WARN  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 1003559 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1003560 WARN  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 1003560 WARN  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 1003561 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 1003561 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 1003561 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 1003562 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_42AE944646D3DDD0-001/control-001/cores/collection1], dataDir=[null]
   [junit4]   2> 1003562 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@91d39bf
   [junit4]   2> 1003562 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_42AE944646D3DDD0-001/control-001/cores/collection1/data
   [junit4]   2> 1003562 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_42AE944646D3DDD0-001/control-001/cores/collection1/data/index/
   [junit4]   2> 1003563 WARN  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_42AE944646D3DDD0-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 1003563 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_42AE944646D3DDD0-001/control-001/cores/collection1/data/index
   [junit4]   2> 1003563 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=38, maxMergeAtOnceExplicit=50, maxMergedSegmentMB=81.46875, floorSegmentMB=1.2275390625, forceMergeDeletesPctAllowed=27.03868587713784, segmentsPerTier=39.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 1003564 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@7fe7f867 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@566573cb),segFN=segments_1,generation=1}
   [junit4]   2> 1003564 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 1003564 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_42AE944646D3DDD0-001/control-001/cores/collection1/data/
   [junit4]   2> 1003564 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_42AE944646D3DDD0-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 1003569 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 1003569 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 1003569 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 1003569 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1003569 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1003569 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 1003569 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1003570 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1003570 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 1003570 WARN  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
   [junit4]   2> 1003573 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1003574 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1003575 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1003576 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1003580 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/mbeans,/admin/threads,/update/json,/admin/luke,/update/json/docs,/update,/admin/system,/dump,/admin/segments,/admin/logging,/replication,/admin/plugins,/schema,standard,/admin/properties,/config,/admin/ping,/admin/file,/get,/update/csv
   [junit4]   2> 1003581 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 1003582 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1003582 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1003582 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1003582 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1003583 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=1273988799, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
   [junit4]   2> 1003583 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@7fe7f867 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@566573cb),segFN=segments_1,generation=1}
   [junit4]   2> 1003583 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 1003584 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@1511f65d[collection1] main
   [junit4]   2> 1003584 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 1003585 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1003585 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 1003585 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 1003585 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 1003585 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1003586 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 1003586 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1003586 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 1003586 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1003586 INFO  (searcherExecutor-2584-thread-1-processing-x:collection1 n:127.0.0.1:48527_kn_u c:control_collection r:core_node1 s:shard1) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@1511f65d[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1003587 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 1003587 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 1003587 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 1003587 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1519651472558522368
   [junit4]   2> 1003589 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 2.0ms to seed version buckets with highest version 1519651472558522368
   [junit4]   2> 1003589 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 1003590 INFO  (coreLoadExecutor-2583-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 1003590 INFO  (coreZkRegister-2577-thread-1-processing-x:collection1 n:127.0.0.1:48527_kn_u c:control_collection r:core_node1 s:shard1) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:48527/kn_u collection:control_collection shard:shard1
   [junit4]   2> 1003591 INFO  (coreZkRegister-2577-thread-1-processing-x:collection1 n:127.0.0.1:48527_kn_u c:control_collection r:core_node1 s:shard1) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 1003597 INFO  (coreZkRegister-2577-thread-1-processing-x:collection1 n:127.0.0.1:48527_kn_u c:control_collection r:core_node1 s:shard1) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94978216905998340-core_node1-n_0000000000
   [junit4]   2> 1003597 INFO  (coreZkRegister-2577-thread-1-processing-x:collection1 n:127.0.0.1:48527_kn_u c:control_collection r:core_node1 s:shard1) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 1003600 INFO  (coreZkRegister-2577-thread-1-processing-x:collection1 n:127.0.0.1:48527_kn_u c:control_collection r:core_node1 s:shard1) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1003600 INFO  (coreZkRegister-2577-thread-1-processing-x:collection1 n:127.0.0.1:48527_kn_u c:control_collection r:core_node1 s:shard1) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1003600 INFO  (coreZkRegister-2577-thread-1-processing-x:collection1 n:127.0.0.1:48527_kn_u c:control_collection r:core_node1 s:shard1) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:48527/kn_u/collection1/
   [junit4]   2> 1003600 INFO  (coreZkRegister-2577-thread-1-processing-x:collection1 n:127.0.0.1:48527_kn_u c:control_collection r:core_node1 s:shard1) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 1003600 INFO  (coreZkRegister-2577-thread-1-processing-x:collection1 n:127.0.0.1:48527_kn_u c:control_collection r:core_node1 s:shard1) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:48527/kn_u/collection1/ has no replicas
   [junit4]   2> 1003600 INFO  (OverseerStateUpdate-94978216905998340-127.0.0.1:48527_kn_u-n_0000000000) [n:127.0.0.1:48527_kn_u    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 1003601 INFO  (coreZkRegister-2577-thread-1-processing-x:collection1 n:127.0.0.1:48527_kn_u c:control_collection r:core_node1 s:shard1) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 1003604 INFO  (coreZkRegister-2577-thread-1-processing-x:collection1 n:127.0.0.1:48527_kn_u c:control_collection r:core_node1 s:shard1) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 1003606 INFO  (coreZkRegister-2577-thread-1-processing-x:collection1 n:127.0.0.1:48527_kn_u c:control_collection r:core_node1 s:shard1) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:48527/kn_u/collection1/ shard1
   [junit4]   2> 1003606 INFO  (OverseerStateUpdate-94978216905998340-127.0.0.1:48527_kn_u-n_0000000000) [n:127.0.0.1:48527_kn_u    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:48527/kn_u",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 1
   [junit4]   2> 1003705 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1003705 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1003707 INFO  (zkCallback-1807-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@4be2b750 name:ZooKeeperConnection Watcher:127.0.0.1:39818/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1003707 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1003708 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1003708 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1003709 INFO  (zkCallback-1805-thread-2-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1003709 INFO  (zkCallback-1805-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1003709 INFO  (zkCallback-1807-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 0)
   [junit4]   2> 1003712 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 1003757 INFO  (coreZkRegister-2577-thread-1-processing-x:collection1 n:127.0.0.1:48527_kn_u c:control_collection r:core_node1 s:shard1) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:48527/kn_u/collection1/ and leader is http://127.0.0.1:48527/kn_u/collection1/
   [junit4]   2> 1003757 INFO  (coreZkRegister-2577-thread-1-processing-x:collection1 n:127.0.0.1:48527_kn_u c:control_collection r:core_node1 s:shard1) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:48527/kn_u
   [junit4]   2> 1003757 INFO  (coreZkRegister-2577-thread-1-processing-x:collection1 n:127.0.0.1:48527_kn_u c:control_collection r:core_node1 s:shard1) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 1003757 INFO  (coreZkRegister-2577-thread-1-processing-x:collection1 n:127.0.0.1:48527_kn_u c:control_collection r:core_node1 s:shard1) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 1003757 INFO  (coreZkRegister-2577-thread-1-processing-x:collection1 n:127.0.0.1:48527_kn_u c:control_collection r:core_node1 s:shard1) [n:127.0.0.1:48527_kn_u c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1003759 INFO  (OverseerStateUpdate-94978216905998340-127.0.0.1:48527_kn_u-n_0000000000) [n:127.0.0.1:48527_kn_u    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "node_name":"127.0.0.1:48527_kn_u",
   [junit4]   2>   "base_url":"http://127.0.0.1:48527/kn_u",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "shard":"shard1"} current state version: 2
   [junit4]   2> 1003760 INFO  (OverseerStateUpdate-94978216905998340-127.0.0.1:48527_kn_u-n_0000000000) [n:127.0.0.1:48527_kn_u    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "node_name":"127.0.0.1:48527_kn_u",
   [junit4]   2>   "base_url":"http://127.0.0.1:48527/kn_u",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "shard":"shard1"}
   [junit4]   2> 1003863 INFO  (zkCallback-1805-thread-2-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1003863 INFO  (zkCallback-1807-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1003863 INFO  (zkCallback-1805-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1004006 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_42AE944646D3DDD0-001/shard-1-001/cores/collection1
   [junit4]   2> 1004007 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_42AE944646D3DDD0-001/shard-1-001
   [junit4]   2> 1004008 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 1004010 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@2f47336d{/kn_u,null,AVAILABLE}
   [junit4]   2> 1004011 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.e.j.s.ServerConnector Started ServerConnector@770422d9{HTTP/1.1}{127.0.0.1:34048}
   [junit4]   2> 1004011 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.e.j.s.Server Started @1006948ms
   [junit4]   2> 1004011 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostPort=34048, solrconfig=solrconfig.xml, hostContext=/kn_u, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_42AE944646D3DDD0-001/tempDir-001/jetty1, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_42AE944646D3DDD0-001/shard-1-001/cores}
   [junit4]   2> 1004011 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@7b3cb2c6
   [junit4]   2> 1004011 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_42AE944646D3DDD0-001/shard-1-001'
   [junit4]   2> 1004012 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1004012 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1004012 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1004013 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1004014 INFO  (zkCallback-1808-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@371fbf99 name:ZooKeeperConnection Watcher:127.0.0.1:39818/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1004014 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1004014 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1004015 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1004015 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_42AE944646D3DDD0-001/shard-1-001/solr.xml
   [junit4]   2> 1004029 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_42AE944646D3DDD0-001/shard-1-001/cores
   [junit4]   2> 1004030 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.CoreContainer New CoreContainer 456153429
   [junit4]   2> 1004030 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_42AE944646D3DDD0-001/shard-1-001]
   [junit4]   2> 1004030 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_42AE944646D3DDD0-001/shard-1-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_42AE944646D3DDD0-001/shard-1-001/lib
   [junit4]   2> 1004030 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1004032 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1004032 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1004032 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1004032 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:39818/solr
   [junit4]   2> 1004032 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 1004032 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1004033 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1004034 INFO  (zkCallback-1810-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@4411ef82 name:ZooKeeperConnection Watcher:127.0.0.1:39818 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1004036 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1004036 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1004039 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [n:127.0.0.1:34048_kn_u    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1004040 INFO  (zkCallback-1811-thread-1-processing-n:127.0.0.1:34048_kn_u) [n:127.0.0.1:34048_kn_u    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@4ec1a51 name:ZooKeeperConnection Watcher:127.0.0.1:39818/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1004040 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [n:127.0.0.1:34048_kn_u    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1004045 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [n:127.0.0.1:34048_kn_u    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1005050 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [n:127.0.0.1:34048_kn_u    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:34048_kn_u
   [junit4]   2> 1005051 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [n:127.0.0.1:34048_kn_u    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:34048_kn_u
   [junit4]   2> 1005052 INFO  (zkCallback-1805-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1005052 INFO  (zkCallback-1811-thread-1-processing-n:127.0.0.1:34048_kn_u) [n:127.0.0.1:34048_kn_u    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1005053 INFO  (zkCallback-1807-thread-1) [    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1005052 INFO  (zkCallback-1805-thread-2-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1005053 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [n:127.0.0.1:34048_kn_u    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1005055 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [n:127.0.0.1:34048_kn_u    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94978216905998344-127.0.0.1:34048_kn_u-n_0000000001
   [junit4]   2> 1005056 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [n:127.0.0.1:34048_kn_u    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/94978216905998340-127.0.0.1:48527_kn_u-n_0000000000 to know if I could be the leader
   [junit4]   2> 1005066 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [n:127.0.0.1:34048_kn_u    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1005066 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [n:127.0.0.1:34048_kn_u    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1005067 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [n:127.0.0.1:34048_kn_u    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_42AE944646D3DDD0-001/shard-1-001/cores
   [junit4]   2> 1005068 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [n:127.0.0.1:34048_kn_u    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {collection=collection1, coreNodeName=, configSetProperties=configsetprops.json, dataDir=data/, transient=false, schema=schema.xml, name=collection1, shard=, loadOnStartup=true, config=solrconfig.xml}
   [junit4]   2> 1005068 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [n:127.0.0.1:34048_kn_u    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_42AE944646D3DDD0-001/shard-1-001/cores/collection1
   [junit4]   2> 1005068 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [n:127.0.0.1:34048_kn_u    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 1005069 INFO  (coreLoadExecutor-2594-thread-1-processing-n:127.0.0.1:34048_kn_u) [n:127.0.0.1:34048_kn_u c:collection1   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 1005069 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [n:127.0.0.1:34048_kn_u    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 1005070 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[42AE944646D3DDD0]) [n:127.0.0.1:34048_kn_u    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1005069 INFO  (coreLoadExecutor-2594-thread-1-processing-n:127.0.0.1:34048_kn_u) [n:127.0.0.1:34048_kn_u c:collection1   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1005072 INFO  (OverseerStateUpdate-94978216905998340-127.0.0.1:48527_kn_u-n_0000000000) [n:127.0.0.1:48527_kn_u    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "node_name":"127.0.0.1:34048_kn_u",
   [junit4]   2>   "base_url":"http://127.0.0.1:34048/kn_u",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "shard":null} current state version: 3
   [junit4]   2> 1005072 INFO  (coreLoadExecutor-2594-thread-1-processing-n:127.0.0.1:34048_kn_u) [n:127.0.0.1:34048_kn_u c:collection1   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 1005072 INFO  (OverseerStateUpdate-94978216905998340-127.0.0.1:48527_kn_u-n_0000000000) [n:127.0.0.1:48527_kn_u    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "node_name":"127.0.0.1:34048_kn_u",
   [junit4]   2>   "base_url":"http://127.0.0.1:34048/kn_u",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "shard":null}
   [junit4]   2> 1005072 INFO  (OverseerStateUpdate-94978216905998340-127.0.0.1:48527_kn_u-n_0000000000) [n:127.0.0.1:48527_kn_u    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 1005072 INFO  (OverseerStateUpdate-94978216905998340-127.0.0.1:48527_kn_u-n_0000000000) [n:127.0.0.1:48527_kn_u    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard2
   [junit4]   2> 1005175 INFO  (zkCallback-1805-thread-1-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1005175 INFO  (zkCallback-1811-thread-1-processing-n:127.0.0.1:34048_kn_u) [n:127.0.0.1:34048_kn_u    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1005175 INFO  (zkCallback-1805-thread-2-processing-n:127.0.0.1:48527_kn_u) [n:127.0.0.1:48527_kn_u    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1005175 INFO  (zkCallback-1807-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1006073 INFO  (coreLoadExecutor-2594-thread-1-processing-n:127.0.0.1:34048_kn_u) [n:127.0.0.1:34048_kn_u c:collection1   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1006073 INFO  (coreLoadExecutor-2594-thread-1-processing-n:127.0.0.1:34048_kn_u) [n:127.0.0.1:34048_kn_u c:collection1   x:collection1] o.a.s.c.ZkController Check for collection zkNode:collection1
   [junit4]   2> 1006074 INFO  (coreLoadExecutor-2594-thread-1-processing-n:127.0.0.1:34048_kn_u) [n:127.0.0.1:34048_kn_u c:collection1   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 1006074 INFO  (coreLoadExecutor-2594-thread-1-processing-n:127.0.0.1:34048_kn_u) [n:127.0.0.1:34048_kn_u c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/collection1
   [junit4]   2> 1006075 INFO  (coreLoadExecutor-2594-thread-1-processing-n:127.0.0.1:34048_kn_u) [n:127.0.0.1:34048_kn_u c:collection1   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1006075 INFO  (coreLoadExecutor-2594-thread-1-processing-n:127.0.0.1:34048_kn_u) [n:127.0.0.1:34048_kn_u c:collection1   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_42AE944646D3DDD0-001/shard-1-001/cores/collection1'
   [junit4]   2> 1006075 INFO  (coreLoadExecutor-2594-thread-1-processing-n:127.0.0.1:34048_kn_u) [n:127.0.0.1:34048_kn_u c:collection1   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1006076 INFO  (coreLoadExecutor-2594-thread-1-processing-n:127.0.0.1:34048_kn_u) [n:127.0.0.1:34048_kn_u c:collection1   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1006078 INFO  (coreLoadExecutor-2594-thread-1-processing-n:127.0.0.1:34048_kn_u) [n:127.0.0.1:34048_kn_u c:collection1   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 1006086 INFO  (coreLoadExecutor-2594-thread-1-processing-n:127.0.0.1:34048_kn_u) [n:127.0.0.1:34048_kn_u c:collection1   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 1006099 INFO  (coreLoadExecutor-2594-thread-1-processing-n:127.0.0.1:34048_kn_u) [n:127.0.0.1:34048_kn_u c:collection1   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.5.0
   [junit4]   2> 1006127 INFO  (coreLoadExecutor-2594-thread-1-processing-n:127.0.0.1:34048_kn_u) [n:127.0.0.1:34048_kn_u c:collection1   x:collection1] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1006136 INFO  (coreLoadExecutor-2594-thread-1-processing-n:127.0.0.1:34048_kn_u) [n:127.0.0.1:34048_kn_u c:collection1   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1006392 INFO  (coreLoadExecutor-2594-thread-1-processing-n:127.0.0.1:34048_kn_u) [n:127.0.0.1:34048_kn_u c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 1006400 INFO  (coreLoadExecutor-2594-thread-1-processing-n:127.0.0.1:34048_kn_u) [n:127.0.0.1:34048_kn_u c:collection1   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 1006402 INFO  (coreLoadExecutor-2594-thread-1-processing-n:127.0.0.1:34048_kn_u) [n:127.0.0.1:34048_kn_u c:collection1   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 1006412 INFO  (coreLoadExecutor-2594-thread-1-processing-n:127.0.0.1:34048_kn_u) [n:127.0.0.1:34048_kn_u c:collection1   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1006417 INFO  (coreLoadExecutor-2594-thread-1-processing-n:127.0.0.1:34048_kn_u) [n:127.0.0.1:34048_kn_u c:collection1   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1006419 INFO  (coreLoadExecutor-2594-thread-1-processing-n:127.0.0.1:34048_kn_u) [n:127.0.0.1:34048_kn_u c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1006420 WARN  (coreLoadExecutor-2594-thread-1-processing-n:127.0.0.1:34048_kn_u) [n:127.0.0.1:34048_kn_u c:collection1   x:collection1] o.a.s.s.OpenExchang

[...truncated too long message...]

9
   [junit4]   2> 3548190 WARN  (Thread-47272) [    ] o.a.s.c.ZkTestServer Watch limit violations: 
   [junit4]   2> Maximum concurrent create/delete watches above limit:
   [junit4]   2> 
   [junit4]   2> 	11	/solr/aliases.json
   [junit4]   2> 	10	/solr/security.json
   [junit4]   2> 	10	/solr/configs/conf1
   [junit4]   2> 
   [junit4]   2> Maximum concurrent data watches above limit:
   [junit4]   2> 
   [junit4]   2> 	17	/solr/collections/collection1/state.json
   [junit4]   2> 	11	/solr/clusterstate.json
   [junit4]   2> 	9	/solr/collections/all_in_lir/state.json
   [junit4]   2> 	6	/solr/overseer_elect/election/94978356697956356-127.0.0.1:34738_zhis%2Fgm-n_0000000000
   [junit4]   2> 	2	/solr/overseer_elect/election/94978356697956377-127.0.0.1:50715_zhis%2Fgm-n_0000000007
   [junit4]   2> 	2	/solr/collections/collection1/leader_elect/shard1/election/94978356697956377-core_node3-n_0000000006
   [junit4]   2> 
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2> 
   [junit4]   2> 	12	/solr/live_nodes
   [junit4]   2> 	11	/solr/collections
   [junit4]   2> 	2	/solr/overseer/queue
   [junit4]   2> 	2	/solr/overseer/collection-queue-work
   [junit4]   2> 
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=LeaderInitiatedRecoveryOnShardRestartTest -Dtests.method=testRestartWithAllInLIR -Dtests.seed=42AE944646D3DDD0 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=ca -Dtests.timezone=US/Mountain -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] FAILURE  413s J2 | LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR <<<
   [junit4]    > Throwable #1: java.lang.AssertionError: There are still nodes recoverying - waited for 330 seconds
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([42AE944646D3DDD0:9C24FA3B7D71BF23]:0)
   [junit4]    > 	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:175)
   [junit4]    > 	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:136)
   [junit4]    > 	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:131)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.waitForRecoveriesToFinish(AbstractFullDistribZkTestBase.java:831)
   [junit4]    > 	at org.apache.solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR(LeaderInitiatedRecoveryOnShardRestartTest.java:104)
   [junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:965)
   [junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:940)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 3548193 INFO  (SUITE-LeaderInitiatedRecoveryOnShardRestartTest-seed#[42AE944646D3DDD0]-worker) [n:127.0.0.1:50715_zhis%2Fgm c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_42AE944646D3DDD0-001
   [junit4]   2> Dec 04, 2015 6:52:03 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=FastCompressingStoredFields(storedFieldsFormat=CompressingStoredFieldsFormat(compressionMode=FAST, chunkSize=28343, maxDocsPerChunk=9, blockSize=516), termVectorsFormat=CompressingTermVectorsFormat(compressionMode=FAST, chunkSize=28343, blockSize=516)), sim=RandomSimilarityProvider(queryNorm=false,coord=yes): {}, locale=ca, timezone=US/Mountain
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_80 (64-bit)/cpus=4,threads=1,free=109854896,total=525860864
   [junit4]   2> NOTE: All tests run in this JVM: [TestStressRecovery, RegexBoostProcessorTest, TestSolrIndexConfig, TestConfigSetsAPIZkFailure, DeleteReplicaTest, SolrCoreCheckLockOnStartupTest, TestCoreDiscovery, SecurityConfHandlerTest, TestFastWriter, TestCSVResponseWriter, TestDistribIDF, HdfsBasicDistributedZkTest, SuggesterWFSTTest, DistributedFacetPivotLongTailTest, TestReloadDeadlock, CurrencyFieldXmlFileTest, RecoveryZkTest, BadIndexSchemaTest, NoCacheHeaderTest, StatelessScriptUpdateProcessorFactoryTest, LukeRequestHandlerTest, QueryParsingTest, TestHighlightDedupGrouping, TestExtendedDismaxParser, TestJettySolrRunner, TestReload, TestManagedStopFilterFactory, LeaderElectionTest, VersionInfoTest, DistribCursorPagingTest, TestSearchPerf, DocExpirationUpdateProcessorFactoryTest, SearchHandlerTest, OverseerRolesTest, TestExactStatsCache, TestNonDefinedSimilarityFactory, TestDistributedStatsComponentCardinality, HdfsThreadLeakTest, DistanceUnitsTest, TestDistributedMissingSort, TestCollapseQParserPlugin, DistribDocExpirationUpdateProcessorTest, RankQueryTest, TestCloudManagedSchema, TestPostingsSolrHighlighter, TestIndexSearcher, BigEndianAscendingWordSerializerTest, XmlUpdateRequestHandlerTest, TestManagedSchemaFieldTypeResource, TestZkChroot, TestRuleBasedAuthorizationPlugin, TestStandardQParsers, RecoveryAfterSoftCommitTest, TestSolrDeletionPolicy1, FacetPivotSmallTest, TestUtils, TestOverriddenPrefixQueryForCustomFieldType, RollingRestartTest, TestWriterPerf, TestRestManager, TestSha256AuthenticationProvider, SuggesterTest, SolrXmlInZkTest, BadComponentTest, SaslZkACLProviderTest, TestGroupingSearch, TestPHPSerializedResponseWriter, TestXIncludeConfig, TestReplicationHandlerBackup, TestSolrQueryParserResource, BJQParserTest, TestRecovery, BufferStoreTest, PreAnalyzedUpdateProcessorTest, FullSolrCloudDistribCmdsTest, TestFreeTextSuggestions, TestRequestForwarding, TestExpandComponent, QueryResultKeyTest, SampleTest, RequestLoggingTest, DirectUpdateHandlerOptimizeTest, OverseerTest, CoreAdminHandlerTest, ClusterStateTest, TestIndexingPerformance, TestJmxMonitoredMap, HighlighterMaxOffsetTest, TestClusterStateMutator, AsyncCallRequestStatusResponseTest, ShardRoutingTest, BasicZkTest, TestReplicationHandler, AutoCommitTest, TestFaceting, DistributedSpellCheckComponentTest, TermVectorComponentDistributedTest, TestStressReorder, TestStressVersions, DistributedTermsComponentTest, SolrCoreTest, PeerSyncTest, ConvertedLegacyTest, TestSort, TestFiltering, TestFunctionQuery, TestLazyCores, DirectUpdateHandlerTest, SoftAutoCommitTest, SolrIndexSplitterTest, SuggesterFSTTest, TestFoldingMultitermQuery, SuggesterTSTTest, WordBreakSolrSpellCheckerTest, FieldMutatingUpdateProcessorTest, TestRemoteStreaming, DebugComponentTest, DisMaxRequestHandlerTest, DirectSolrSpellCheckerTest, DocumentAnalysisRequestHandlerTest, TestOmitPositions, TermsComponentTest, TermVectorComponentTest, FieldAnalysisRequestHandlerTest, FastVectorHighlighterTest, LoggingHandlerTest, TestCollationField, UpdateRequestProcessorFactoryTest, MBeansHandlerTest, BinaryUpdateRequestHandlerTest, TestBinaryResponseWriter, HighlighterConfigTest, UpdateParamsTest, AlternateDirectoryTest, TestQuerySenderNoQuery, SolrIndexConfigTest, TestSolrDeletionPolicy2, MultiTermTest, TestDocSet, TestBinaryField, NumericFieldsTest, TestElisionMultitermQuery, TestConfig, TestFuzzyAnalyzedSuggestions, ExternalFileFieldSortTest, TestSolrCoreProperties, DirectSolrConnectionTest, TestLuceneMatchVersion, TestPhraseSuggestions, SpellPossibilityIteratorTest, TestCharFilters, TestSweetSpotSimilarityFactory, TestDFRSimilarityFactory, TestPerFieldSimilarity, TestLMJelinekMercerSimilarityFactory, TestIBSimilarityFactory, TimeZoneUtilsTest, ResourceLoaderTest, ScriptEngineTest, OpenExchangeRatesOrgProviderTest, URLClassifyProcessorTest, PluginInfoTest, TestFastLRUCache, DateMathParserTest, PreAnalyzedFieldTest, TestSystemIdResolver, PrimUtilsTest, TestSuggestSpellingConverter, DateFieldTest, RAMDirectoryFactoryTest, TestSolrJ, ZkNodePropsTest, SliceStateTest, FileUtilsTest, CircularListTest, SolrTestCaseJ4Test, TestCrossCoreJoin, TestCursorMarkWithoutUniqueKey, TestSimpleTrackingShardHandler, AsyncMigrateRouteKeyTest, ConfigSetsAPITest, DeleteShardTest, LeaderInitiatedRecoveryOnShardRestartTest]
   [junit4] Completed [439/552] on J2 in 413.76s, 1 test, 1 failure <<< FAILURES!

[...truncated 359 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:799: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:736: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 552 suites (7 ignored), 2185 tests, 1 error, 1 failure, 33 ignored (18 assumptions) [seed: 42AE944646D3DDD0]

Total time: 209 minutes 2 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1035 - Still Failing

[Apache Jenkins Server <je...@builds.apache.org>]

Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1035/

2 tests failed.
FAILED:  org.apache.solr.hadoop.MorphlineBasicMiniMRTest.mrRun

Error Message:
Failed on local exception: java.io.IOException: Connection reset by peer; Host Details : local host is: "lucene1-us-west/10.41.0.5"; destination host is: "lucene1-us-west.apache.org":53477; 

Stack Trace:
java.io.IOException: Failed on local exception: java.io.IOException: Connection reset by peer; Host Details : local host is: "lucene1-us-west/10.41.0.5"; destination host is: "lucene1-us-west.apache.org":53477; 
	at __randomizedtesting.SeedInfo.seed([562707F3EC0A0F3D:5875B3FDED9C3D32]:0)
	at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:772)
	at org.apache.hadoop.ipc.Client.call(Client.java:1472)
	at org.apache.hadoop.ipc.Client.call(Client.java:1399)
	at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:232)
	at com.sun.proxy.$Proxy109.getClusterMetrics(Unknown Source)
	at org.apache.hadoop.yarn.api.impl.pb.client.ApplicationClientProtocolPBClientImpl.getClusterMetrics(ApplicationClientProtocolPBClientImpl.java:202)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:187)
	at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)
	at com.sun.proxy.$Proxy110.getClusterMetrics(Unknown Source)
	at org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.getYarnClusterMetrics(YarnClientImpl.java:461)
	at org.apache.hadoop.mapred.ResourceMgrDelegate.getClusterMetrics(ResourceMgrDelegate.java:151)
	at org.apache.hadoop.mapred.YARNRunner.getClusterMetrics(YARNRunner.java:179)
	at org.apache.hadoop.mapreduce.Cluster.getClusterStatus(Cluster.java:246)
	at org.apache.hadoop.mapred.JobClient$3.run(JobClient.java:719)
	at org.apache.hadoop.mapred.JobClient$3.run(JobClient.java:717)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
	at org.apache.hadoop.mapred.JobClient.getClusterStatus(JobClient.java:717)
	at org.apache.solr.hadoop.MapReduceIndexerTool.run(MapReduceIndexerTool.java:645)
	at org.apache.solr.hadoop.MapReduceIndexerTool.run(MapReduceIndexerTool.java:608)
	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
	at org.apache.solr.hadoop.MorphlineBasicMiniMRTest.mrRun(MorphlineBasicMiniMRTest.java:363)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.IOException: Connection reset by peer
	at sun.nio.ch.FileDispatcherImpl.read0(Native Method)
	at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
	at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223)
	at sun.nio.ch.IOUtil.read(IOUtil.java:197)
	at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:384)
	at org.apache.hadoop.net.SocketInputStream$Reader.performIO(SocketInputStream.java:57)
	at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:142)
	at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161)
	at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131)
	at java.io.FilterInputStream.read(FilterInputStream.java:133)
	at java.io.FilterInputStream.read(FilterInputStream.java:133)
	at org.apache.hadoop.ipc.Client$Connection$PingInputStream.read(Client.java:513)
	at java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
	at java.io.BufferedInputStream.read(BufferedInputStream.java:254)
	at java.io.DataInputStream.readInt(DataInputStream.java:387)
	at org.apache.hadoop.ipc.Client$Connection.receiveRpcResponse(Client.java:1071)
	at org.apache.hadoop.ipc.Client$Connection.run(Client.java:966)


FAILED:  org.apache.solr.hadoop.MorphlineGoLiveMiniMRTest.test

Error Message:
Failed on local exception: java.io.IOException: Connection reset by peer; Host Details : local host is: "lucene1-us-west/10.41.0.5"; destination host is: "lucene1-us-west.apache.org":50361; 

Stack Trace:
java.io.IOException: Failed on local exception: java.io.IOException: Connection reset by peer; Host Details : local host is: "lucene1-us-west/10.41.0.5"; destination host is: "lucene1-us-west.apache.org":50361; 
	at __randomizedtesting.SeedInfo.seed([562707F3EC0A0F3D:DE73382942F662C5]:0)
	at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:772)
	at org.apache.hadoop.ipc.Client.call(Client.java:1472)
	at org.apache.hadoop.ipc.Client.call(Client.java:1399)
	at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:232)
	at com.sun.proxy.$Proxy111.getClusterMetrics(Unknown Source)
	at org.apache.hadoop.yarn.api.impl.pb.client.ApplicationClientProtocolPBClientImpl.getClusterMetrics(ApplicationClientProtocolPBClientImpl.java:202)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:187)
	at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)
	at com.sun.proxy.$Proxy112.getClusterMetrics(Unknown Source)
	at org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.getYarnClusterMetrics(YarnClientImpl.java:461)
	at org.apache.hadoop.mapred.ResourceMgrDelegate.getClusterMetrics(ResourceMgrDelegate.java:151)
	at org.apache.hadoop.mapred.YARNRunner.getClusterMetrics(YARNRunner.java:179)
	at org.apache.hadoop.mapreduce.Cluster.getClusterStatus(Cluster.java:246)
	at org.apache.hadoop.mapred.JobClient$3.run(JobClient.java:719)
	at org.apache.hadoop.mapred.JobClient$3.run(JobClient.java:717)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
	at org.apache.hadoop.mapred.JobClient.getClusterStatus(JobClient.java:717)
	at org.apache.solr.hadoop.MapReduceIndexerTool.run(MapReduceIndexerTool.java:645)
	at org.apache.solr.hadoop.MapReduceIndexerTool.run(MapReduceIndexerTool.java:608)
	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
	at org.apache.solr.hadoop.MorphlineGoLiveMiniMRTest.test(MorphlineGoLiveMiniMRTest.java:402)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:965)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:940)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.IOException: Connection reset by peer
	at sun.nio.ch.FileDispatcherImpl.read0(Native Method)
	at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
	at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223)
	at sun.nio.ch.IOUtil.read(IOUtil.java:197)
	at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:384)
	at org.apache.hadoop.net.SocketInputStream$Reader.performIO(SocketInputStream.java:57)
	at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:142)
	at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161)
	at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131)
	at java.io.FilterInputStream.read(FilterInputStream.java:133)
	at java.io.FilterInputStream.read(FilterInputStream.java:133)
	at org.apache.hadoop.ipc.Client$Connection$PingInputStream.read(Client.java:513)
	at java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
	at java.io.BufferedInputStream.read(BufferedInputStream.java:254)
	at java.io.DataInputStream.readInt(DataInputStream.java:387)
	at org.apache.hadoop.ipc.Client$Connection.receiveRpcResponse(Client.java:1071)
	at org.apache.hadoop.ipc.Client$Connection.run(Client.java:966)




Build Log:
[...truncated 18504 lines...]
   [junit4] Suite: org.apache.solr.hadoop.MorphlineBasicMiniMRTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/init-core-data-001
   [junit4]   2> 0    INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false)
   [junit4]   2> 1027 WARN  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.u.NativeCodeLoader Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
   [junit4]   1> Formatting using clusterid: testClusterID
   [junit4]   2> 1126 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.n.FSNamesystem No KeyProvider found.
   [junit4]   2> 1168 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.n.FSNamesystem fsLock is fair:true
   [junit4]   2> 1246 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.c.C.deprecation hadoop.configured.node.mapping is deprecated. Instead, use net.topology.configured.node.mapping
   [junit4]   2> 1246 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.b.DatanodeManager dfs.block.invalidate.limit=1000
   [junit4]   2> 1246 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.b.DatanodeManager dfs.namenode.datanode.registration.ip-hostname-check=true
   [junit4]   2> 1253 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.b.BlockManager dfs.namenode.startup.delay.block.deletion.sec is set to 000:00:00:00.000
   [junit4]   2> 1256 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.b.BlockManager The block deletion will start around 2015 ??? 03 23:01:28
   [junit4]   2> 1260 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.u.GSet Computing capacity for map BlocksMap
   [junit4]   2> 1261 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.u.GSet VM type       = 64-bit
   [junit4]   2> 1263 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.u.GSet 2.0% max memory 455 MB = 9.1 MB
   [junit4]   2> 1263 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.u.GSet capacity      = 2^20 = 1048576 entries
   [junit4]   2> 1293 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.b.BlockManager dfs.block.access.token.enable=false
   [junit4]   2> 1294 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.b.BlockManager defaultReplication         = 2
   [junit4]   2> 1294 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.b.BlockManager maxReplication             = 512
   [junit4]   2> 1294 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.b.BlockManager minReplication             = 1
   [junit4]   2> 1294 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.b.BlockManager maxReplicationStreams      = 2
   [junit4]   2> 1295 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.b.BlockManager shouldCheckForEnoughRacks  = false
   [junit4]   2> 1295 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.b.BlockManager replicationRecheckInterval = 3000
   [junit4]   2> 1295 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.b.BlockManager encryptDataTransfer        = false
   [junit4]   2> 1295 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.b.BlockManager maxNumBlocksToLog          = 1000
   [junit4]   2> 1304 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.n.FSNamesystem fsOwner             = jenkins (auth:SIMPLE)
   [junit4]   2> 1304 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.n.FSNamesystem supergroup          = supergroup
   [junit4]   2> 1305 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.n.FSNamesystem isPermissionEnabled = true
   [junit4]   2> 1305 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.n.FSNamesystem HA Enabled: false
   [junit4]   2> 1312 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.n.FSNamesystem Append Enabled: true
   [junit4]   2> 1779 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.u.GSet Computing capacity for map INodeMap
   [junit4]   2> 1779 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.u.GSet VM type       = 64-bit
   [junit4]   2> 1779 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.u.GSet 1.0% max memory 455 MB = 4.5 MB
   [junit4]   2> 1779 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.u.GSet capacity      = 2^19 = 524288 entries
   [junit4]   2> 1780 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.n.NameNode Caching file names occuring more than 10 times
   [junit4]   2> 1796 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.u.GSet Computing capacity for map cachedBlocks
   [junit4]   2> 1796 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.u.GSet VM type       = 64-bit
   [junit4]   2> 1796 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.u.GSet 0.25% max memory 455 MB = 1.1 MB
   [junit4]   2> 1796 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.u.GSet capacity      = 2^17 = 131072 entries
   [junit4]   2> 1799 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.n.FSNamesystem dfs.namenode.safemode.threshold-pct = 0.9990000128746033
   [junit4]   2> 1799 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.n.FSNamesystem dfs.namenode.safemode.min.datanodes = 0
   [junit4]   2> 1799 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.n.FSNamesystem dfs.namenode.safemode.extension     = 0
   [junit4]   2> 1802 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.n.FSNamesystem Retry cache on namenode is enabled
   [junit4]   2> 1802 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.n.FSNamesystem Retry cache will use 0.03 of total heap and retry cache entry expiry time is 600000 millis
   [junit4]   2> 1809 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.u.GSet Computing capacity for map NameNodeRetryCache
   [junit4]   2> 1809 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.u.GSet VM type       = 64-bit
   [junit4]   2> 1809 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.u.GSet 0.029999999329447746% max memory 455 MB = 139.8 KB
   [junit4]   2> 1809 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.u.GSet capacity      = 2^14 = 16384 entries
   [junit4]   2> 1820 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.n.NNConf ACLs enabled? false
   [junit4]   2> 1820 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.n.NNConf XAttrs enabled? true
   [junit4]   2> 1820 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.n.NNConf Maximum size of an xattr: 16384
   [junit4]   2> 1924 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.n.FSImage Allocated new BlockPoolId: BP-1268983991-10.41.0.5-1449176489074
   [junit4]   2> 1943 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.c.Storage Storage directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/name1 has been successfully formatted.
   [junit4]   2> 1947 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.c.Storage Storage directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/name2 has been successfully formatted.
   [junit4]   2> 2187 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.n.NNStorageRetentionManager Going to retain 1 images with txid >= 0
   [junit4]   2> 2192 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.n.NameNode createNameNode []
   [junit4]   2> 2251 WARN  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
   [junit4]   2> 2352 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.m.i.MetricsSystemImpl Scheduled snapshot period at 10 second(s).
   [junit4]   2> 2353 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.m.i.MetricsSystemImpl NameNode metrics system started
   [junit4]   2> 2356 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.n.NameNode fs.defaultFS is hdfs://127.0.0.1:0
   [junit4]   2> 2432 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.DFSUtil Starting Web-server for hdfs at: http://localhost:0
   [junit4]   2> 2599 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.m.log Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
   [junit4]   2> 2630 WARN  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 2669 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.HttpServer2 Added global filter 'safety' (class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter)
   [junit4]   2> 2673 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.HttpServer2 Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context hdfs
   [junit4]   2> 2673 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.HttpServer2 Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context static
   [junit4]   2> 2674 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.HttpServer2 Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs
   [junit4]   2> 2727 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.HttpServer2 Added filter 'org.apache.hadoop.hdfs.web.AuthFilter' (class=org.apache.hadoop.hdfs.web.AuthFilter)
   [junit4]   2> 2730 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.HttpServer2 addJerseyResourcePackage: packageName=org.apache.hadoop.hdfs.server.namenode.web.resources;org.apache.hadoop.hdfs.web.resources, pathSpec=/webhdfs/v1/*
   [junit4]   2> 2778 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.HttpServer2 Jetty bound to port 35720
   [junit4]   2> 2778 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 2859 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/hdfs to ./temp/Jetty_localhost_35720_hdfs____.cdvm0z/webapp
   [junit4]   2> 3151 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 5362 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:35720
   [junit4]   2> 5378 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.n.FSNamesystem No KeyProvider found.
   [junit4]   2> 5378 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.n.FSNamesystem fsLock is fair:true
   [junit4]   2> 5379 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.b.DatanodeManager dfs.block.invalidate.limit=1000
   [junit4]   2> 5379 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.b.DatanodeManager dfs.namenode.datanode.registration.ip-hostname-check=true
   [junit4]   2> 5380 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.b.BlockManager dfs.namenode.startup.delay.block.deletion.sec is set to 000:00:00:00.000
   [junit4]   2> 5380 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.b.BlockManager The block deletion will start around 2015 ??? 03 23:01:32
   [junit4]   2> 5380 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.u.GSet Computing capacity for map BlocksMap
   [junit4]   2> 5381 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.u.GSet VM type       = 64-bit
   [junit4]   2> 5381 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.u.GSet 2.0% max memory 455 MB = 9.1 MB
   [junit4]   2> 5381 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.u.GSet capacity      = 2^20 = 1048576 entries
   [junit4]   2> 5382 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.b.BlockManager dfs.block.access.token.enable=false
   [junit4]   2> 5382 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.b.BlockManager defaultReplication         = 2
   [junit4]   2> 5382 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.b.BlockManager maxReplication             = 512
   [junit4]   2> 5383 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.b.BlockManager minReplication             = 1
   [junit4]   2> 5383 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.b.BlockManager maxReplicationStreams      = 2
   [junit4]   2> 5383 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.b.BlockManager shouldCheckForEnoughRacks  = false
   [junit4]   2> 5383 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.b.BlockManager replicationRecheckInterval = 3000
   [junit4]   2> 5383 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.b.BlockManager encryptDataTransfer        = false
   [junit4]   2> 5383 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.b.BlockManager maxNumBlocksToLog          = 1000
   [junit4]   2> 5384 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.n.FSNamesystem fsOwner             = jenkins (auth:SIMPLE)
   [junit4]   2> 5384 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.n.FSNamesystem supergroup          = supergroup
   [junit4]   2> 5384 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.n.FSNamesystem isPermissionEnabled = true
   [junit4]   2> 5385 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.n.FSNamesystem HA Enabled: false
   [junit4]   2> 5385 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.n.FSNamesystem Append Enabled: true
   [junit4]   2> 5386 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.u.GSet Computing capacity for map INodeMap
   [junit4]   2> 5386 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.u.GSet VM type       = 64-bit
   [junit4]   2> 5386 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.u.GSet 1.0% max memory 455 MB = 4.5 MB
   [junit4]   2> 5386 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.u.GSet capacity      = 2^19 = 524288 entries
   [junit4]   2> 5387 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.n.NameNode Caching file names occuring more than 10 times
   [junit4]   2> 5387 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.u.GSet Computing capacity for map cachedBlocks
   [junit4]   2> 5387 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.u.GSet VM type       = 64-bit
   [junit4]   2> 5387 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.u.GSet 0.25% max memory 455 MB = 1.1 MB
   [junit4]   2> 5388 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.u.GSet capacity      = 2^17 = 131072 entries
   [junit4]   2> 5388 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.n.FSNamesystem dfs.namenode.safemode.threshold-pct = 0.9990000128746033
   [junit4]   2> 5388 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.n.FSNamesystem dfs.namenode.safemode.min.datanodes = 0
   [junit4]   2> 5388 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.n.FSNamesystem dfs.namenode.safemode.extension     = 0
   [junit4]   2> 5388 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.n.FSNamesystem Retry cache on namenode is enabled
   [junit4]   2> 5389 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.n.FSNamesystem Retry cache will use 0.03 of total heap and retry cache entry expiry time is 600000 millis
   [junit4]   2> 5389 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.u.GSet Computing capacity for map NameNodeRetryCache
   [junit4]   2> 5389 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.u.GSet VM type       = 64-bit
   [junit4]   2> 5389 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.u.GSet 0.029999999329447746% max memory 455 MB = 139.8 KB
   [junit4]   2> 5389 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.u.GSet capacity      = 2^14 = 16384 entries
   [junit4]   2> 5392 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.n.NNConf ACLs enabled? false
   [junit4]   2> 5392 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.n.NNConf XAttrs enabled? true
   [junit4]   2> 5393 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.n.NNConf Maximum size of an xattr: 16384
   [junit4]   2> 5403 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.c.Storage Lock on /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/name1/in_use.lock acquired by nodename 32282@lucene1-us-west
   [junit4]   2> 5410 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.c.Storage Lock on /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/name2/in_use.lock acquired by nodename 32282@lucene1-us-west
   [junit4]   2> 5414 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.n.FileJournalManager Recovering unfinalized segments in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/name1/current
   [junit4]   2> 5415 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.n.FileJournalManager Recovering unfinalized segments in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/name2/current
   [junit4]   2> 5415 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.n.FSImage No edit log streams selected.
   [junit4]   2> 5452 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.n.FSImageFormatPBINode Loading 1 INodes.
   [junit4]   2> 5467 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.n.FSImageFormatProtobuf Loaded FSImage in 0 seconds.
   [junit4]   2> 5467 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.n.FSImage Loaded image for txid 0 from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/name1/current/fsimage_0000000000000000000
   [junit4]   2> 5474 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.n.FSNamesystem Need to save fs image? false (staleImage=false, haEnabled=false, isRollingUpgrade=false)
   [junit4]   2> 5475 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.n.FSEditLog Starting log segment at 1
   [junit4]   2> 5549 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.n.NameCache initialized with 0 entries 0 lookups
   [junit4]   2> 5549 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.n.FSNamesystem Finished loading FSImage in 156 msecs
   [junit4]   2> 5993 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.n.NameNode RPC server is binding to localhost:0
   [junit4]   2> 6008 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.i.CallQueueManager Using callQueue class java.util.concurrent.LinkedBlockingQueue
   [junit4]   2> 6035 INFO  (Socket Reader #1 for port 43259) [    ] o.a.h.i.Server Starting Socket Reader #1 for port 43259
   [junit4]   2> 6092 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.n.NameNode Clients are to use localhost:43259 to access this namenode/service.
   [junit4]   2> 6095 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.n.FSNamesystem Registered FSNamesystemState MBean
   [junit4]   2> 6123 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.n.FSNamesystem Number of blocks under construction: 0
   [junit4]   2> 6123 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.n.FSNamesystem Number of blocks under construction: 0
   [junit4]   2> 6123 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.n.FSNamesystem initializing replication queues
   [junit4]   2> 6124 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.StateChange STATE* Leaving safe mode after 0 secs
   [junit4]   2> 6125 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.StateChange STATE* Network topology has 0 racks and 0 datanodes
   [junit4]   2> 6125 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.StateChange STATE* UnderReplicatedBlocks has 0 blocks
   [junit4]   2> 6131 INFO  (Replication Queue Initializer) [    ] o.a.h.h.s.b.BlockManager Total number of blocks            = 0
   [junit4]   2> 6132 INFO  (Replication Queue Initializer) [    ] o.a.h.h.s.b.BlockManager Number of invalid blocks          = 0
   [junit4]   2> 6132 INFO  (Replication Queue Initializer) [    ] o.a.h.h.s.b.BlockManager Number of under-replicated blocks = 0
   [junit4]   2> 6132 INFO  (Replication Queue Initializer) [    ] o.a.h.h.s.b.BlockManager Number of  over-replicated blocks = 0
   [junit4]   2> 6132 INFO  (Replication Queue Initializer) [    ] o.a.h.h.s.b.BlockManager Number of blocks being written    = 0
   [junit4]   2> 6132 INFO  (Replication Queue Initializer) [    ] o.a.h.h.StateChange STATE* Replication Queue initialization scan for invalid, over- and under-replicated blocks completed in 6 msec
   [junit4]   2> 6222 INFO  (IPC Server listener on 43259) [    ] o.a.h.i.Server IPC Server listener on 43259: starting
   [junit4]   2> 6222 INFO  (IPC Server Responder) [    ] o.a.h.i.Server IPC Server Responder: starting
   [junit4]   2> 6231 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.n.NameNode NameNode RPC up at: localhost/127.0.0.1:43259
   [junit4]   2> 6231 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.n.FSNamesystem Starting services required for active state
   [junit4]   2> 6255 INFO  (CacheReplicationMonitor(676048482)) [    ] o.a.h.h.s.b.CacheReplicationMonitor Starting CacheReplicationMonitor with interval 30000 milliseconds
   [junit4]   2> 6256 INFO  (CacheReplicationMonitor(676048482)) [    ] o.a.h.h.s.b.CacheReplicationMonitor Rescanning after 17636790587 milliseconds
   [junit4]   2> 6256 INFO  (CacheReplicationMonitor(676048482)) [    ] o.a.h.h.s.b.CacheReplicationMonitor Scanned 0 directive(s) and 0 block(s) in 0 millisecond(s).
   [junit4]   2> 6258 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.MiniDFSCluster Starting DataNode 0 with dfs.datanode.data.dir: [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data1,[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data2
   [junit4]   2> 6344 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.m.i.MetricsSystemImpl DataNode metrics system started (again)
   [junit4]   2> 6346 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.d.DataNode Configured hostname is 127.0.0.1
   [junit4]   2> 6354 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.d.DataNode Starting DataNode with maxLockedMemory = 0
   [junit4]   2> 6364 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.d.DataNode Opened streaming server at /127.0.0.1:34131
   [junit4]   2> 6368 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.d.DataNode Balancing bandwith is 1048576 bytes/s
   [junit4]   2> 6368 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.d.DataNode Number threads for balancing is 5
   [junit4]   2> 6383 WARN  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 6385 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.HttpServer2 Added global filter 'safety' (class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter)
   [junit4]   2> 6386 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.HttpServer2 Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context datanode
   [junit4]   2> 6386 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.HttpServer2 Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context static
   [junit4]   2> 6386 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.HttpServer2 Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs
   [junit4]   2> 6393 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.HttpServer2 addJerseyResourcePackage: packageName=org.apache.hadoop.hdfs.server.datanode.web.resources;org.apache.hadoop.hdfs.web.resources, pathSpec=/webhdfs/v1/*
   [junit4]   2> 6394 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.HttpServer2 Jetty bound to port 56245
   [junit4]   2> 6394 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 6417 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_56245_datanode____.mzamnb/webapp
   [junit4]   2> 6604 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 7108 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:56245
   [junit4]   2> 7110 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.d.DataNode dnUserName = jenkins
   [junit4]   2> 7110 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.d.DataNode supergroup = supergroup
   [junit4]   2> 7145 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.i.CallQueueManager Using callQueue class java.util.concurrent.LinkedBlockingQueue
   [junit4]   2> 7146 INFO  (Socket Reader #1 for port 34967) [    ] o.a.h.i.Server Starting Socket Reader #1 for port 34967
   [junit4]   2> 7163 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.d.DataNode Opened IPC server at /127.0.0.1:34967
   [junit4]   2> 7177 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.d.DataNode Refresh request received for nameservices: null
   [junit4]   2> 7181 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.d.DataNode Starting BPOfferServices for nameservices: <default>
   [junit4]   2> 7209 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:43259) [    ] o.a.h.h.s.d.DataNode Block pool <registering> (Datanode Uuid unassigned) service to localhost/127.0.0.1:43259 starting to offer service
   [junit4]   2> 7225 INFO  (IPC Server Responder) [    ] o.a.h.i.Server IPC Server Responder: starting
   [junit4]   2> 7234 INFO  (IPC Server listener on 34967) [    ] o.a.h.i.Server IPC Server listener on 34967: starting
   [junit4]   2> 7239 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.MiniDFSCluster Starting DataNode 1 with dfs.datanode.data.dir: [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data3,[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data4
   [junit4]   2> 7397 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.m.i.MetricsSystemImpl DataNode metrics system started (again)
   [junit4]   2> 7398 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.d.DataNode Configured hostname is 127.0.0.1
   [junit4]   2> 7398 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.d.DataNode Starting DataNode with maxLockedMemory = 0
   [junit4]   2> 7399 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.d.DataNode Opened streaming server at /127.0.0.1:34571
   [junit4]   2> 7399 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.d.DataNode Balancing bandwith is 1048576 bytes/s
   [junit4]   2> 7399 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.d.DataNode Number threads for balancing is 5
   [junit4]   2> 7401 WARN  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 7403 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.HttpServer2 Added global filter 'safety' (class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter)
   [junit4]   2> 7411 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.HttpServer2 Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context datanode
   [junit4]   2> 7411 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.HttpServer2 Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs
   [junit4]   2> 7411 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.HttpServer2 Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context static
   [junit4]   2> 7413 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.HttpServer2 addJerseyResourcePackage: packageName=org.apache.hadoop.hdfs.server.datanode.web.resources;org.apache.hadoop.hdfs.web.resources, pathSpec=/webhdfs/v1/*
   [junit4]   2> 7413 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.HttpServer2 Jetty bound to port 51484
   [junit4]   2> 7414 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 7468 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_51484_datanode____.uist8l/webapp
   [junit4]   2> 7774 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 7994 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:43259) [    ] o.a.h.h.s.c.Storage DataNode version: -56 and NameNode layout version: -60
   [junit4]   2> 7998 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:43259) [    ] o.a.h.h.s.c.Storage Lock on /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data1/in_use.lock acquired by nodename 32282@lucene1-us-west
   [junit4]   2> 8001 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:43259) [    ] o.a.h.h.s.c.Storage Storage directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data1 is not formatted for BP-1268983991-10.41.0.5-1449176489074
   [junit4]   2> 8001 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:43259) [    ] o.a.h.h.s.c.Storage Formatting ...
   [junit4]   2> 8008 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:43259) [    ] o.a.h.h.s.c.Storage Lock on /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data2/in_use.lock acquired by nodename 32282@lucene1-us-west
   [junit4]   2> 8009 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:43259) [    ] o.a.h.h.s.c.Storage Storage directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data2 is not formatted for BP-1268983991-10.41.0.5-1449176489074
   [junit4]   2> 8009 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:43259) [    ] o.a.h.h.s.c.Storage Formatting ...
   [junit4]   2> 8168 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:43259) [    ] o.a.h.h.s.c.Storage Analyzing storage directories for bpid BP-1268983991-10.41.0.5-1449176489074
   [junit4]   2> 8168 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:43259) [    ] o.a.h.h.s.c.Storage Locking is disabled
   [junit4]   2> 8170 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:43259) [    ] o.a.h.h.s.c.Storage Storage directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data1/current/BP-1268983991-10.41.0.5-1449176489074 is not formatted.
   [junit4]   2> 8170 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:43259) [    ] o.a.h.h.s.c.Storage Formatting ...
   [junit4]   2> 8171 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:43259) [    ] o.a.h.h.s.c.Storage Formatting block pool BP-1268983991-10.41.0.5-1449176489074 directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data1/current/BP-1268983991-10.41.0.5-1449176489074/current
   [junit4]   2> 8174 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:43259) [    ] o.a.h.h.s.c.Storage Locking is disabled
   [junit4]   2> 8175 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:43259) [    ] o.a.h.h.s.c.Storage Storage directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data2/current/BP-1268983991-10.41.0.5-1449176489074 is not formatted.
   [junit4]   2> 8175 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:43259) [    ] o.a.h.h.s.c.Storage Formatting ...
   [junit4]   2> 8175 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:43259) [    ] o.a.h.h.s.c.Storage Formatting block pool BP-1268983991-10.41.0.5-1449176489074 directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data2/current/BP-1268983991-10.41.0.5-1449176489074/current
   [junit4]   2> 8178 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:43259) [    ] o.a.h.h.s.c.Storage Restored 0 block files from trash.
   [junit4]   2> 8182 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:51484
   [junit4]   2> 8185 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:43259) [    ] o.a.h.h.s.c.Storage Restored 0 block files from trash.
   [junit4]   2> 8185 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.d.DataNode dnUserName = jenkins
   [junit4]   2> 8185 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.d.DataNode supergroup = supergroup
   [junit4]   2> 8185 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.i.CallQueueManager Using callQueue class java.util.concurrent.LinkedBlockingQueue
   [junit4]   2> 8187 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:43259) [    ] o.a.h.h.s.d.DataNode Setting up storage: nsid=1179921729;bpid=BP-1268983991-10.41.0.5-1449176489074;lv=-56;nsInfo=lv=-60;cid=testClusterID;nsid=1179921729;c=0;bpid=BP-1268983991-10.41.0.5-1449176489074;dnuuid=null
   [junit4]   2> 8190 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:43259) [    ] o.a.h.h.s.d.DataNode Generated and persisted new Datanode UUID a1a5894e-b5bd-430b-b6b0-af8a7bb588de
   [junit4]   2> 8191 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.d.DataNode Opened IPC server at /127.0.0.1:49663
   [junit4]   2> 8200 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.d.DataNode Refresh request received for nameservices: null
   [junit4]   2> 8201 INFO  (Socket Reader #1 for port 49663) [    ] o.a.h.i.Server Starting Socket Reader #1 for port 49663
   [junit4]   2> 8213 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [    ] o.a.h.h.s.d.DataNode Starting BPOfferServices for nameservices: <default>
   [junit4]   2> 8217 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:43259) [    ] o.a.h.h.s.d.DataNode Block pool <registering> (Datanode Uuid unassigned) service to localhost/127.0.0.1:43259 starting to offer service
   [junit4]   2> 8217 INFO  (IPC Server Responder) [    ] o.a.h.i.Server IPC Server Responder: starting
   [junit4]   2> 8218 INFO  (IPC Server listener on 49663) [    ] o.a.h.i.Server IPC Server listener on 49663: starting
   [junit4]   2> 8230 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:43259) [    ] o.a.h.h.s.c.Storage DataNode version: -56 and NameNode layout version: -60
   [junit4]   2> 8234 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:43259) [    ] o.a.h.h.s.c.Storage Lock on /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data3/in_use.lock acquired by nodename 32282@lucene1-us-west
   [junit4]   2> 8235 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:43259) [    ] o.a.h.h.s.c.Storage Storage directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data3 is not formatted for BP-1268983991-10.41.0.5-1449176489074
   [junit4]   2> 8235 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:43259) [    ] o.a.h.h.s.c.Storage Formatting ...
   [junit4]   2> 8240 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:43259) [    ] o.a.h.h.s.c.Storage Lock on /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data4/in_use.lock acquired by nodename 32282@lucene1-us-west
   [junit4]   2> 8250 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:43259) [    ] o.a.h.h.s.c.Storage Storage directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data4 is not formatted for BP-1268983991-10.41.0.5-1449176489074
   [junit4]   2> 8250 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:43259) [    ] o.a.h.h.s.c.Storage Formatting ...
   [junit4]   2> 8309 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:43259) [    ] o.a.h.h.s.d.f.i.FsDatasetImpl Added new volume: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data1/current
   [junit4]   2> 8341 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:43259) [    ] o.a.h.h.s.d.f.i.FsDatasetImpl Added volume - /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data1/current, StorageType: DISK
   [junit4]   2> 8342 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:43259) [    ] o.a.h.h.s.d.f.i.FsDatasetImpl Added new volume: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data2/current
   [junit4]   2> 8342 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:43259) [    ] o.a.h.h.s.d.f.i.FsDatasetImpl Added volume - /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data2/current, StorageType: DISK
   [junit4]   2> 8391 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:43259) [    ] o.a.h.h.s.d.f.i.FsDatasetImpl Registered FSDatasetState MBean
   [junit4]   2> 8402 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:43259) [    ] o.a.h.h.s.d.DirectoryScanner Periodic Directory Tree Verification scan starting at 1449189806630 with interval 21600000
   [junit4]   2> 8403 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:43259) [    ] o.a.h.h.s.d.f.i.FsDatasetImpl Adding block pool BP-1268983991-10.41.0.5-1449176489074
   [junit4]   2> 8403 INFO  (Thread-89) [    ] o.a.h.h.s.d.f.i.FsDatasetImpl Scanning block pool BP-1268983991-10.41.0.5-1449176489074 on volume /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data1/current...
   [junit4]   2> 8404 INFO  (Thread-90) [    ] o.a.h.h.s.d.f.i.FsDatasetImpl Scanning block pool BP-1268983991-10.41.0.5-1449176489074 on volume /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data2/current...
   [junit4]   2> 8516 INFO  (Thread-89) [    ] o.a.h.h.s.d.f.i.FsDatasetImpl Time taken to scan block pool BP-1268983991-10.41.0.5-1449176489074 on /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data1/current: 106ms
   [junit4]   2> 8516 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:43259) [    ] o.a.h.h.s.c.Storage Analyzing storage directories for bpid BP-1268983991-10.41.0.5-1449176489074
   [junit4]   2> 8517 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:43259) [    ] o.a.h.h.s.c.Storage Locking is disabled
   [junit4]   2> 8518 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:43259) [    ] o.a.h.h.s.c.Storage Storage directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data3/current/BP-1268983991-10.41.0.5-1449176489074 is not formatted.
   [junit4]   2> 8518 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:43259) [    ] o.a.h.h.s.c.Storage Formatting ...
   [junit4]   2> 8518 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:43259) [    ] o.a.h.h.s.c.Storage Formatting block pool BP-1268983991-10.41.0.5-1449176489074 directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data3/current/BP-1268983991-10.41.0.5-1449176489074/current
   [junit4]   2> 8521 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:43259) [    ] o.a.h.h.s.c.Storage Locking is disabled
   [junit4]   2> 8522 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:43259) [    ] o.a.h.h.s.c.Storage Storage directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data4/current/BP-1268983991-10.41.0.5-1449176489074 is not formatted.
   [junit4]   2> 8522 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:43259) [    ] o.a.h.h.s.c.Storage Formatting ...
   [junit4]   2> 8522 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:43259) [    ] o.a.h.h.s.c.Storage Formatting block pool BP-1268983991-10.41.0.5-1449176489074 directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data4/current/BP-1268983991-10.41.0.5-1449176489074/current
   [junit4]   2> 8525 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:43259) [    ] o.a.h.h.s.c.Storage Restored 0 block files from trash.
   [junit4]   2> 8525 INFO  (Thread-90) [    ] o.a.h.h.s.d.f.i.FsDatasetImpl Time taken to scan block pool BP-1268983991-10.41.0.5-1449176489074 on /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data2/current: 96ms
   [junit4]   2> 8525 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:43259) [    ] o.a.h.h.s.d.f.i.FsDatasetImpl Total time to scan all replicas for block pool BP-1268983991-10.41.0.5-1449176489074: 122ms
   [junit4]   2> 8525 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:43259) [    ] o.a.h.h.s.c.Storage Restored 0 block files from trash.
   [junit4]   2> 8537 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:43259) [    ] o.a.h.h.s.d.DataNode Setting up storage: nsid=1179921729;bpid=BP-1268983991-10.41.0.5-1449176489074;lv=-56;nsInfo=lv=-60;cid=testClusterID;nsid=1179921729;c=0;bpid=BP-1268983991-10.41.0.5-1449176489074;dnuuid=null
   [junit4]   2> 8541 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:43259) [    ] o.a.h.h.s.d.DataNode Generated and persisted new Datanode UUID 59e613d4-abae-4102-9e3d-5075dd9f97bc
   [junit4]   2> 8542 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_562707F3EC0A0F3D-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:43259) [    ] o.a.h.h.s.d.f.i.FsDatasetImpl Added new volume: /x1/jenkin

[...truncated too long message...]

r:core_node5 x:collection1] o.a.h.h.s.n.FSNamesystem Stopping services started for standby state
   [junit4]   2> 83287 INFO  (SUITE-MorphlineGoLiveMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [n:127.0.0.1:47248_v_y c:collection1 s:shard1 r:core_node5 x:collection1] o.m.log Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0
   [junit4]   2> 83389 INFO  (SUITE-MorphlineGoLiveMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [n:127.0.0.1:47248_v_y c:collection1 s:shard1 r:core_node5 x:collection1] o.a.h.m.i.MetricsSystemImpl Stopping JobHistoryServer metrics system...
   [junit4]   2> 83391 ERROR (SUITE-MorphlineGoLiveMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [n:127.0.0.1:47248_v_y c:collection1 s:shard1 r:core_node5 x:collection1] o.a.h.m.l.MethodMetric Error invoking method getBlocksTotal
   [junit4]   2> java.lang.reflect.InvocationTargetException
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   [junit4]   2> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2> 	at java.lang.reflect.Method.invoke(Method.java:606)
   [junit4]   2> 	at org.apache.hadoop.metrics2.lib.MethodMetric$2.snapshot(MethodMetric.java:111)
   [junit4]   2> 	at org.apache.hadoop.metrics2.lib.MethodMetric.snapshot(MethodMetric.java:144)
   [junit4]   2> 	at org.apache.hadoop.metrics2.lib.MetricsRegistry.snapshot(MetricsRegistry.java:387)
   [junit4]   2> 	at org.apache.hadoop.metrics2.lib.MetricsSourceBuilder$1.getMetrics(MetricsSourceBuilder.java:79)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.getMetrics(MetricsSourceAdapter.java:195)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.updateJmxCache(MetricsSourceAdapter.java:172)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.getMBeanInfo(MetricsSourceAdapter.java:151)
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getClassName(DefaultMBeanServerInterceptor.java:1804)
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.safeGetClassName(DefaultMBeanServerInterceptor.java:1595)
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.checkMBeanPermission(DefaultMBeanServerInterceptor.java:1813)
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.exclusiveUnregisterMBean(DefaultMBeanServerInterceptor.java:430)
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.unregisterMBean(DefaultMBeanServerInterceptor.java:415)
   [junit4]   2> 	at com.sun.jmx.mbeanserver.JmxMBeanServer.unregisterMBean(JmxMBeanServer.java:546)
   [junit4]   2> 	at org.apache.hadoop.metrics2.util.MBeans.unregister(MBeans.java:81)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.stopMBeans(MetricsSourceAdapter.java:227)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.stop(MetricsSourceAdapter.java:212)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.stopSources(MetricsSystemImpl.java:461)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.stop(MetricsSystemImpl.java:212)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.shutdown(MetricsSystemImpl.java:592)
   [junit4]   2> 	at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.shutdownInstance(DefaultMetricsSystem.java:72)
   [junit4]   2> 	at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.shutdown(DefaultMetricsSystem.java:68)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.metrics.NameNodeMetrics.shutdown(NameNodeMetrics.java:145)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.NameNode.stop(NameNode.java:822)
   [junit4]   2> 	at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1720)
   [junit4]   2> 	at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1699)
   [junit4]   2> 	at org.apache.solr.hadoop.MorphlineGoLiveMiniMRTest.teardownClass(MorphlineGoLiveMiniMRTest.java:236)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   [junit4]   2> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2> 	at java.lang.reflect.Method.invoke(Method.java:606)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:834)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> Caused by: java.lang.NullPointerException
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.blockmanagement.BlocksMap.size(BlocksMap.java:198)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.getTotalBlocks(BlockManager.java:3291)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlocksTotal(FSNamesystem.java:6223)
   [junit4]   2> 	... 53 more
   [junit4]   2> 83405 INFO  (SUITE-MorphlineGoLiveMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [n:127.0.0.1:47248_v_y c:collection1 s:shard1 r:core_node5 x:collection1] o.a.h.m.i.MetricsSystemImpl JobHistoryServer metrics system stopped.
   [junit4]   2> 83405 INFO  (SUITE-MorphlineGoLiveMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [n:127.0.0.1:47248_v_y c:collection1 s:shard1 r:core_node5 x:collection1] o.a.h.m.i.MetricsSystemImpl JobHistoryServer metrics system shutdown complete.
   [junit4]   2> 83406 INFO  (SUITE-MorphlineGoLiveMiniMRTest-seed#[562707F3EC0A0F3D]-worker) [n:127.0.0.1:47248_v_y c:collection1 s:shard1 r:core_node5 x:collection1] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J1/temp/solr.hadoop.MorphlineGoLiveMiniMRTest_562707F3EC0A0F3D-001
   [junit4]   2> NOTE: test params are: codec=Lucene54, sim=RandomSimilarityProvider(queryNorm=false,coord=no): {}, locale=sq_AL, timezone=Etc/GMT0
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_80 (64-bit)/cpus=4,threads=8,free=120967240,total=343408640
   [junit4]   2> NOTE: All tests run in this JVM: [MorphlineReducerTest, MapReduceIndexerToolArgumentParserTest, MorphlineGoLiveMiniMRTest]
   [junit4] Completed [7/7] on J1 in 76.59s, 2 tests, 1 error <<< FAILURES!

[...truncated 12 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:799: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:736: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:253: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:467: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 7 suites, 35 tests, 2 errors, 1 ignored [seed: 562707F3EC0A0F3D]

Total time: 215 minutes 55 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1034 - Still Failing

[Apache Jenkins Server <je...@builds.apache.org>]

Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1034/

1 tests failed.
FAILED:  org.apache.lucene.search.TestGeoPointQuery.testRandomBig

Error Message:
Java heap space

Stack Trace:
java.lang.OutOfMemoryError: Java heap space
	at __randomizedtesting.SeedInfo.seed([6B493944E699B5AC:EC1E44CB77C0C92C]:0)
	at org.apache.lucene.util.ArrayUtil.grow(ArrayUtil.java:354)
	at org.apache.lucene.codecs.memory.DirectPostingsFormat$DirectField.<init>(DirectPostingsFormat.java:363)
	at org.apache.lucene.codecs.memory.DirectPostingsFormat$DirectFields.<init>(DirectPostingsFormat.java:131)
	at org.apache.lucene.codecs.memory.DirectPostingsFormat.fieldsProducer(DirectPostingsFormat.java:115)
	at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsReader.<init>(PerFieldPostingsFormat.java:261)
	at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat.fieldsProducer(PerFieldPostingsFormat.java:341)
	at org.apache.lucene.index.SegmentCoreReaders.<init>(SegmentCoreReaders.java:104)
	at org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:65)
	at org.apache.lucene.index.CheckIndex.checkIndex(CheckIndex.java:655)
	at org.apache.lucene.util.TestUtil.checkIndex(TestUtil.java:278)
	at org.apache.lucene.store.MockDirectoryWrapper.close(MockDirectoryWrapper.java:757)
	at org.apache.lucene.util.IOUtils.close(IOUtils.java:97)
	at org.apache.lucene.util.IOUtils.close(IOUtils.java:84)
	at org.apache.lucene.util.BaseGeoPointTestCase.verify(BaseGeoPointTestCase.java:766)
	at org.apache.lucene.util.BaseGeoPointTestCase.doTestRandom(BaseGeoPointTestCase.java:399)
	at org.apache.lucene.util.BaseGeoPointTestCase.testRandomBig(BaseGeoPointTestCase.java:327)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)




Build Log:
[...truncated 8761 lines...]
   [junit4] Suite: org.apache.lucene.search.TestGeoPointQuery
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestGeoPointQuery -Dtests.method=testRandomBig -Dtests.seed=6B493944E699B5AC -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=ms -Dtests.timezone=Europe/Lisbon -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
   [junit4] ERROR   2438s J1 | TestGeoPointQuery.testRandomBig <<<
   [junit4]    > Throwable #1: java.lang.OutOfMemoryError: Java heap space
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([6B493944E699B5AC:EC1E44CB77C0C92C]:0)
   [junit4]    > 	at org.apache.lucene.util.ArrayUtil.grow(ArrayUtil.java:354)
   [junit4]    > 	at org.apache.lucene.codecs.memory.DirectPostingsFormat$DirectField.<init>(DirectPostingsFormat.java:363)
   [junit4]    > 	at org.apache.lucene.codecs.memory.DirectPostingsFormat$DirectFields.<init>(DirectPostingsFormat.java:131)
   [junit4]    > 	at org.apache.lucene.codecs.memory.DirectPostingsFormat.fieldsProducer(DirectPostingsFormat.java:115)
   [junit4]    > 	at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsReader.<init>(PerFieldPostingsFormat.java:261)
   [junit4]    > 	at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat.fieldsProducer(PerFieldPostingsFormat.java:341)
   [junit4]    > 	at org.apache.lucene.index.SegmentCoreReaders.<init>(SegmentCoreReaders.java:104)
   [junit4]    > 	at org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:65)
   [junit4]    > 	at org.apache.lucene.index.CheckIndex.checkIndex(CheckIndex.java:655)
   [junit4]    > 	at org.apache.lucene.util.TestUtil.checkIndex(TestUtil.java:278)
   [junit4]    > 	at org.apache.lucene.store.MockDirectoryWrapper.close(MockDirectoryWrapper.java:757)
   [junit4]    > 	at org.apache.lucene.util.IOUtils.close(IOUtils.java:97)
   [junit4]    > 	at org.apache.lucene.util.IOUtils.close(IOUtils.java:84)
   [junit4]    > 	at org.apache.lucene.util.BaseGeoPointTestCase.verify(BaseGeoPointTestCase.java:766)
   [junit4]    > 	at org.apache.lucene.util.BaseGeoPointTestCase.doTestRandom(BaseGeoPointTestCase.java:399)
   [junit4]    > 	at org.apache.lucene.util.BaseGeoPointTestCase.testRandomBig(BaseGeoPointTestCase.java:327)
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/sandbox/test/J1/temp/lucene.search.TestGeoPointQuery_6B493944E699B5AC-001
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene54): {string=FST50, id=FST50, point=PostingsFormat(name=Direct)}, docValues:{id=DocValuesFormat(name=SimpleText), point=DocValuesFormat(name=Asserting)}, sim=DefaultSimilarity, locale=ms, timezone=Europe/Lisbon
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_80 (64-bit)/cpus=4,threads=1,free=331360376,total=398983168
   [junit4]   2> NOTE: All tests run in this JVM: [TestSlowFuzzyQuery2, TestGeoPointQuery]
   [junit4] Completed [19/20] on J1 in 2892.41s, 23 tests, 1 error <<< FAILURES!

[...truncated 4 lines...]
   [junit4] JVM J1: stdout was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/sandbox/test/temp/junit4-J1-20151202_125238_127.sysout
   [junit4] >>> JVM J1: stdout (verbatim) ----
   [junit4] java.lang.OutOfMemoryError: Java heap space
   [junit4] Dumping heap to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/heapdumps/java_pid29086.hprof ...
   [junit4] Heap dump file created [555886611 bytes in 4.516 secs]
   [junit4] <<< JVM J1: EOF ----

[...truncated 10 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build.xml:471: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:2248: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/module-build.xml:58: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 20 suites, 142 tests, 1 error, 1 ignored (1 assumption) [seed: 6B493944E699B5AC]

Total time: 259 minutes 32 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1033 - Still Failing

[Apache Jenkins Server <je...@builds.apache.org>]

Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1033/

3 tests failed.
FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=22727, name=zkCallback-553-thread-15, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=22727, name=zkCallback-553-thread-15, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.CollectionsAPIDistributedZkTest

Error Message:
116 threads leaked from SUITE scope at org.apache.solr.cloud.CollectionsAPIDistributedZkTest:     1) Thread[id=22487, name=searcherExecutor-3572-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)    2) Thread[id=19958, name=qtp129174011-19958-selector-ServerConnectorManager@9c681d/1, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102)         at org.eclipse.jetty.io.SelectorManager$ManagedSelector.select(SelectorManager.java:600)         at org.eclipse.jetty.io.SelectorManager$ManagedSelector.run(SelectorManager.java:549)         at org.eclipse.jetty.util.thread.NonBlockingThread.run(NonBlockingThread.java:52)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)         at java.lang.Thread.run(Thread.java:745)    3) Thread[id=20335, name=searcherExecutor-1760-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)    4) Thread[id=19965, name=Scheduler-1630664890, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1090)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)    5) Thread[id=19981, name=searcherExecutor-1498-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)    6) Thread[id=20257, name=searcherExecutor-1666-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)    7) Thread[id=22474, name=searcherExecutor-3577-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)    8) Thread[id=22980, name=zkCallback-606-thread-12-processing-n:127.0.0.1:60958_, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.common.cloud.ZkCmdExecutor.retryDelay(ZkCmdExecutor.java:108)         at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:76)         at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:342)         at org.apache.solr.cloud.ZkController.getLeaderProps(ZkController.java:1028)         at org.apache.solr.cloud.ZkController.getLeaderProps(ZkController.java:1014)         at org.apache.solr.cloud.ZkController.waitForLeaderToSeeDownState(ZkController.java:1597)         at org.apache.solr.cloud.ZkController.registerAllCoresAsDown(ZkController.java:443)         at org.apache.solr.cloud.ZkController.access$100(ZkController.java:89)         at org.apache.solr.cloud.ZkController$1.command(ZkController.java:272)         at org.apache.solr.common.cloud.ConnectionManager$1.update(ConnectionManager.java:158)         at org.apache.solr.common.cloud.DefaultConnectionStrategy.reconnect(DefaultConnectionStrategy.java:56)         at org.apache.solr.common.cloud.ConnectionManager.process(ConnectionManager.java:132)         at org.apache.solr.common.cloud.SolrZkClient$3$1.run(SolrZkClient.java:264)         at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)         at java.util.concurrent.FutureTask.run(FutureTask.java:262)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)    9) Thread[id=22440, name=searcherExecutor-3525-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   10) Thread[id=22301, name=qtp129174011-22301, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)         at java.lang.Thread.run(Thread.java:745)   11) Thread[id=22229, name=searcherExecutor-3399-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   12) Thread[id=20435, name=qtp2011327979-20435-selector-ServerConnectorManager@1be9f792/0, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102)         at org.eclipse.jetty.io.SelectorManager$ManagedSelector.select(SelectorManager.java:600)         at org.eclipse.jetty.io.SelectorManager$ManagedSelector.run(SelectorManager.java:549)         at org.eclipse.jetty.util.thread.NonBlockingThread.run(NonBlockingThread.java:52)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)         at java.lang.Thread.run(Thread.java:745)   13) Thread[id=22563, name=searcherExecutor-3606-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   14) Thread[id=22228, name=searcherExecutor-3404-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   15) Thread[id=20666, name=searcherExecutor-1988-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   16) Thread[id=20437, name=qtp2011327979-20437-selector-ServerConnectorManager@1be9f792/1, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102)         at org.eclipse.jetty.io.SelectorManager$ManagedSelector.select(SelectorManager.java:600)         at org.eclipse.jetty.io.SelectorManager$ManagedSelector.run(SelectorManager.java:549)         at org.eclipse.jetty.util.thread.NonBlockingThread.run(NonBlockingThread.java:52)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)         at java.lang.Thread.run(Thread.java:745)   17) Thread[id=22465, name=searcherExecutor-3503-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   18) Thread[id=20469, name=Thread-17402, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Object.wait(Native Method)         at java.lang.Object.wait(Object.java:503)         at org.apache.solr.core.CloserThread.run(CoreContainer.java:1238)   19) Thread[id=20534, name=searcherExecutor-1902-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   20) Thread[id=21184, name=qtp1015471035-21184, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.cloud.ZkController.getLeaderProps(ZkController.java:1041)         at org.apache.solr.cloud.ZkController.getLeaderProps(ZkController.java:1014)         at org.apache.solr.cloud.ZkController.getLeader(ZkController.java:970)         at org.apache.solr.cloud.ZkController.register(ZkController.java:909)         at org.apache.solr.cloud.ZkController.register(ZkController.java:852)         at org.apache.solr.core.ZkContainer$2.run(ZkContainer.java:183)         at org.apache.solr.core.ZkContainer.registerInZk(ZkContainer.java:212)         at org.apache.solr.core.CoreContainer.registerCore(CoreContainer.java:696)         at org.apache.solr.core.CoreContainer.create(CoreContainer.java:820)         at org.apache.solr.core.CoreContainer.create(CoreContainer.java:750)         at org.apache.solr.handler.admin.CoreAdminHandler.handleCreateAction(CoreAdminHandler.java:617)         at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestInternal(CoreAdminHandler.java:212)         at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:192)         at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:151)         at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:660)         at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:436)         at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:221)         at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:180)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:109)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.eclipse.jetty.servlets.UserAgentFilter.doFilter(UserAgentFilter.java:83)         at org.eclipse.jetty.servlets.GzipFilter.doFilter(GzipFilter.java:300)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)         at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:221)         at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)         at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)         at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)         at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)         at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)         at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)         at org.eclipse.jetty.server.Server.handle(Server.java:499)         at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)         at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)         at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)         at java.lang.Thread.run(Thread.java:745)   21) Thread[id=20024, name=Scheduler-141426940, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1090)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   22) Thread[id=20285, name=searcherExecutor-1718-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   23) Thread[id=22438, name=searcherExecutor-3517-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   24) Thread[id=20016, name=qtp1015471035-20016-selector-ServerConnectorManager@26c9cef5/0, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102)         at org.eclipse.jetty.io.SelectorManager$ManagedSelector.select(SelectorManager.java:600)         at org.eclipse.jetty.io.SelectorManager$ManagedSelector.run(SelectorManager.java:549)         at org.eclipse.jetty.util.thread.NonBlockingThread.run(NonBlockingThread.java:52)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)         at java.lang.Thread.run(Thread.java:745)   25) Thread[id=22386, name=searcherExecutor-3445-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   26) Thread[id=22066, name=qtp1015471035-22066, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)         at java.lang.Thread.run(Thread.java:745)   27) Thread[id=20645, name=searcherExecutor-1978-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   28) Thread[id=20151, name=searcherExecutor-1600-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   29) Thread[id=22481, name=searcherExecutor-3568-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   30) Thread[id=20139, name=searcherExecutor-1587-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   31) Thread[id=20442, name=Scheduler-674380372, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1090)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   32) Thread[id=22215, name=searcherExecutor-3393-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   33) Thread[id=20036, name=Thread-17187, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Object.wait(Native Method)         at java.lang.Object.wait(Object.java:503)         at org.apache.solr.core.CloserThread.run(CoreContainer.java:1238)   34) Thread[id=20350, name=searcherExecutor-1772-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   35) Thread[id=22427, name=searcherExecutor-3489-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   36) Thread[id=20543, name=searcherExecutor-1907-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   37) Thread[id=22290, name=qtp1015471035-22290, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.cloud.ZkController.getLeaderProps(ZkController.java:1041)         at org.apache.solr.cloud.ZkController.getLeaderProps(ZkController.java:1014)         at org.apache.solr.cloud.ZkController.getLeader(ZkController.java:970)         at org.apache.solr.cloud.ZkController.register(ZkController.java:909)         at org.apache.solr.cloud.ZkController.register(ZkController.java:852)         at org.apache.solr.core.ZkContainer$2.run(ZkContainer.java:183)         at org.apache.solr.core.ZkContainer.registerInZk(ZkContainer.java:212)         at org.apache.solr.core.CoreContainer.registerCore(CoreContainer.java:696)         at org.apache.solr.core.CoreContainer.create(CoreContainer.java:820)         at org.apache.solr.core.CoreContainer.create(CoreContainer.java:750)         at org.apache.solr.handler.admin.CoreAdminHandler.handleCreateAction(CoreAdminHandler.java:617)         at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestInternal(CoreAdminHandler.java:212)         at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:192)         at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:151)         at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:660)         at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:436)         at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:221)         at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:180)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:109)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.eclipse.jetty.servlets.UserAgentFilter.doFilter(UserAgentFilter.java:83)         at org.eclipse.jetty.servlets.GzipFilter.doFilter(GzipFilter.java:300)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)         at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:221)         at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)         at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)         at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)         at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)         at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)         at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)         at org.eclipse.jetty.server.Server.handle(Server.java:499)         at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)         at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)         at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)         at java.lang.Thread.run(Thread.java:745)   38) Thread[id=22394, name=searcherExecutor-3470-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   39) Thread[id=20053, name=Scheduler-969142536, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1090)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   40) Thread[id=22231, name=searcherExecutor-3406-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   41) Thread[id=22470, name=searcherExecutor-3578-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   42) Thread[id=22326, name=qtp1015471035-22326, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)         at java.lang.Thread.run(Thread.java:745)   43) Thread[id=22323, name=qtp129174011-22323, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)         at java.lang.Thread.run(Thread.java:745)   44) Thread[id=23081, name=zkCallback-606-thread-12-processing-n:127.0.0.1:60958_-SendThread(127.0.0.1:50840), state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)         at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:940)         at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1003)   45) Thread[id=20241, name=searcherExecutor-1668-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   46) Thread[id=20509, name=searcherExecutor-1879-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   47) Thread[id=20487, name=Scheduler-849255626, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1090)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   48) Thread[id=20671, name=searcherExecutor-1998-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   49) Thread[id=20025, name=org.eclipse.jetty.server.session.HashSessionManager@5821be24Timer, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1090)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   50) Thread[id=22505, name=searcherExecutor-3573-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   51) Thread[id=20171, name=searcherExecutor-1636-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   52) Thread[id=20040, name=searcherExecutor-1520-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   53) Thread[id=20149, name=searcherExecutor-1597-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   54) Thread[id=20263, name=searcherExecutor-1694-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   55) Thread[id=20045, name=Scheduler-2069119407, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1079)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   56) Thread[id=22264, name=qtp129174011-22264, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)         at java.lang.Thread.run(Thread.java:745)   57) Thread[id=20544, name=searcherExecutor-1897-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   58) Thread[id=22496, name=searcherExecutor-3562-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   59) Thread[id=20152, name=searcherExecutor-1595-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   60) Thread[id=22391, name=searcherExecutor-3465-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   61) Thread[id=20061, name=searcherExecutor-1532-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   62) Thread[id=22430, name=searcherExecutor-3483-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   63) Thread[id=20249, name=searcherExecutor-1669-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   64) Thread[id=21237, name=qtp2011327979-21237, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.common.cloud.ZkCmdExecutor.retryDelay(ZkCmdExecutor.java:108)         at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:76)         at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:342)         at org.apache.solr.cloud.ZkController.getLeaderProps(ZkController.java:1028)         at org.apache.solr.cloud.ZkController.getLeaderProps(ZkController.java:1014)         at org.apache.solr.cloud.ZkController.getLeader(ZkController.java:970)         at org.apache.solr.cloud.ZkController.register(ZkController.java:909)         at org.apache.solr.cloud.ZkController.register(ZkController.java:852)         at org.apache.solr.core.ZkContainer$2.run(ZkContainer.java:183)         at org.apache.solr.core.ZkContainer.registerInZk(ZkContainer.java:212)         at org.apache.solr.core.CoreContainer.registerCore(CoreContainer.java:696)         at org.apache.solr.core.CoreContainer.create(CoreContainer.java:820)         at org.apache.solr.core.CoreContainer.create(CoreContainer.java:750)         at org.apache.solr.handler.admin.CoreAdminHandler.handleCreateAction(CoreAdminHandler.java:617)         at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestInternal(CoreAdminHandler.java:212)         at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:192)         at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:151)         at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:660)         at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:436)         at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:221)         at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:180)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:109)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.eclipse.jetty.servlets.UserAgentFilter.doFilter(UserAgentFilter.java:83)         at org.eclipse.jetty.servlets.GzipFilter.doFilter(GzipFilter.java:300)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)         at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:221)         at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)         at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)         at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)         at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)         at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)         at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)         at org.eclipse.jetty.server.Server.handle(Server.java:499)         at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)         at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)         at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)         at java.lang.Thread.run(Thread.java:745)   65) Thread[id=23051, name=zkCallback-606-thread-12-processing-n:127.0.0.1:60958_-EventThread, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.common.cloud.ZkCmdExecutor.retryDelay(ZkCmdExecutor.java:108)         at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:76)         at org.apache.solr.common.cloud.SolrZkClient.create(SolrZkClient.java:377)         at org.apache.solr.cloud.DistributedQueue.offer(DistributedQueue.java:250)         at org.apache.solr.cloud.ZkController.publish(ZkController.java:1228)         at org.apache.solr.cloud.ZkController.publish(ZkController.java:1140)         at org.apache.solr.cloud.ZkController.registerAllCoresAsDown(ZkController.java:393)         at org.apache.solr.cloud.ZkController.access$100(ZkController.java:89)         at org.apache.solr.cloud.ZkController$1.command(ZkController.java:272)         at org.apache.solr.common.cloud.ConnectionManager$1.update(ConnectionManager.java:158)         at org.apache.solr.common.cloud.DefaultConnectionStrategy.reconnect(DefaultConnectionStrategy.java:56)         at org.apache.solr.common.cloud.ConnectionManager.process(ConnectionManager.java:132)         at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)         at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)   66) Thread[id=20547, name=searcherExecutor-1908-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   67) Thread[id=20481, name=searcherExecutor-1866-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   68) Thread[id=20438, name=qtp2011327979-20438-acceptor-0@61111b68-ServerConnector@1d7ca08f{SSL-http/1.1}{127.0.0.1:60958}, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250)         at org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:377)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:500)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)         at java.lang.Thread.run(Thread.java:745)   69) Thread[id=22569, name=searcherExecutor-3623-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   70) Thread[id=20531, name=searcherExecutor-1896-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   71) Thread[id=20258, name=searcherExecutor-1689-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   72) Thread[id=20166, name=searcherExecutor-1630-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   73) Thread[id=22371, name=searcherExecutor-3433-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   74) Thread[id=20662, name=searcherExecutor-1983-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   75) Thread[id=20506, name=searcherExecutor-1881-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   76) Thread[id=22222, name=searcherExecutor-3395-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   77) Thread[id=20499, name=searcherExecutor-1876-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   78) Thread[id=19957, name=qtp129174011-19957-selector-ServerConnectorManager@9c681d/0, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102)         at org.eclipse.jetty.io.SelectorManager$ManagedSelector.select(SelectorManager.java:600)         at org.eclipse.jetty.io.SelectorManager$ManagedSelector.run(SelectorManager.java:549)         at org.eclipse.jetty.util.thread.NonBlockingThread.run(NonBlockingThread.java:52)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)         at java.lang.Thread.run(Thread.java:745)   79) Thread[id=23082, name=zkCallback-606-thread-12-processing-n:127.0.0.1:60958_-EventThread, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:494)   80) Thread[id=22538, name=searcherExecutor-3604-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   81) Thread[id=20363, name=searcherExecutor-1791-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   82) Thread[id=20225, name=searcherExecutor-1660-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   83) Thread[id=20533, name=searcherExecutor-1906-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   84) Thread[id=20668, name=searcherExecutor-1989-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   85) Thread[id=20399, name=searcherExecutor-1828-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   86) Thread[id=20491, name=searcherExecutor-1871-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   87) Thread[id=22385, name=searcherExecutor-3446-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   88) Thread[id=20340, name=searcherExecutor-1761-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   89) Thread[id=22320, name=qtp129174011-22320, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)         at java.lang.Thread.run(Thread.java:745)   90) Thread[id=22434, name=searcherExecutor-3506-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   91) Thread[id=20018, name=qtp1015471035-20018-acceptor-0@12c77bc6-ServerConnector@b324ab8{SSL-http/1.1}{127.0.0.1:51129}, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250)         at org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:377)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:500)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)         at java.lang.Thread.run(Thread.java:745)   92) Thread[id=22543, name=searcherExecutor-3622-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   93) Thread[id=22541, name=searcherExecutor-3614-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   94) Thread[id=19959, name=qtp129174011-19959-acceptor-0@5b272816-ServerConnector@1c77483b{SSL-http/1.1}{127.0.0.1:45593}, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250)         at org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:377)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:500)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)         at java.lang.Thread.run(Thread.java:745)   95) Thread[id=22418, name=searcherExecutor-3493-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   96) Thread[id=22469, name=searcherExecutor-3556-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   97) Thread[id=22200, name=qtp2011327979-22200, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)         at java.lang.Thread.run(Thread.java:745)   98) Thread[id=20505, name=searcherExecutor-1877-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   99) Thread[id=22080, name=qtp129174011-22080, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)         at java.lang.Thread.run(Thread.java:745)   100) Thread[id=22373, name=searcherExecutor-3434-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   101) Thread[id=19966, name=org.eclipse.jetty.server.session.HashSessionManager@

[...truncated too long message...]

lectorImpl.select(SelectorImpl.java:102)
   [junit4]    >         at org.eclipse.jetty.io.SelectorManager$ManagedSelector.select(SelectorManager.java:600)
   [junit4]    >         at org.eclipse.jetty.io.SelectorManager$ManagedSelector.run(SelectorManager.java:549)
   [junit4]    >         at org.eclipse.jetty.util.thread.NonBlockingThread.run(NonBlockingThread.java:52)
   [junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
   [junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >   90) Thread[id=22427, name=searcherExecutor-3489-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
   [junit4]    >         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >   91) Thread[id=19966, name=org.eclipse.jetty.server.session.HashSessionManager@75f1a51eTimer, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
   [junit4]    >         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1090)
   [junit4]    >         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >   92) Thread[id=20393, name=searcherExecutor-1825-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
   [junit4]    >         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >   93) Thread[id=20543, name=searcherExecutor-1907-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
   [junit4]    >         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >   94) Thread[id=20642, name=searcherExecutor-1973-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
   [junit4]    >         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >   95) Thread[id=20053, name=Scheduler-969142536, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
   [junit4]    >         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1090)
   [junit4]    >         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >   96) Thread[id=22394, name=searcherExecutor-3470-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
   [junit4]    >         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([B01AFA1951047BBB]:0)
   [junit4] Completed [463/552] on J1 in 3276.63s, 1 test, 3 errors <<< FAILURES!

[...truncated 261 lines...]
   [junit4] JVM J1: stdout was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J1-20151201_084124_734.sysout
   [junit4] >>> JVM J1: stdout (verbatim) ----
   [junit4] java.lang.OutOfMemoryError: GC overhead limit exceeded
   [junit4] Dumping heap to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/heapdumps/java_pid30103.hprof ...
   [junit4] Heap dump file created [611582236 bytes in 10.024 secs]
   [junit4] <<< JVM J1: EOF ----

   [junit4] JVM J1: stderr was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J1-20151201_084124_734.syserr
   [junit4] >>> JVM J1: stderr (verbatim) ----
   [junit4] 
   [junit4] Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "zkCallback-553-thread-13"
   [junit4] <<< JVM J1: EOF ----

[...truncated 17 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 552 suites (7 ignored), 2183 tests, 2 suite-level errors, 1 error, 303 ignored (18 assumptions) [seed: B01AFA1951047BBB]

Total time: 213 minutes 32 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1032 - Still Failing

[Apache Jenkins Server <je...@builds.apache.org>]

Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1032/

1 tests failed.
FAILED:  org.apache.solr.schema.TestCloudSchemaless.test

Error Message:
QUERY FAILED: xpath=/response/arr[@name='fields']/lst/str[@name='name'][.='newTestFieldInt447']  request=/schema/fields?wt=xml  response=<?xml version="1.0" encoding="UTF-8"?> <response>  <lst name="responseHeader">   <int name="status">0</int>   <int name="QTime">172</int> </lst> <arr name="fields">   <lst>     <str name="name">_root_</str>     <str name="type">string</str>     <bool name="multiValued">false</bool>     <bool name="stored">true</bool>     <bool name="indexed">true</bool>   </lst>   <lst>     <str name="name">_version_</str>     <str name="type">long</str>     <bool name="stored">true</bool>     <bool name="indexed">true</bool>   </lst>   <lst>     <str name="name">constantField</str>     <str name="type">tdouble</str>   </lst>   <lst>     <str name="name">id</str>     <str name="type">string</str>     <bool name="required">true</bool>     <bool name="multiValued">false</bool>     <bool name="stored">true</bool>     <bool name="indexed">true</bool>     <bool name="uniqueKey">true</bool>   </lst>   <lst>     <str name="name">newTestFieldInt0</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt1</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt10</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt100</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt101</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt102</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt103</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt104</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt105</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt106</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt107</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt108</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt109</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt11</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt110</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt111</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt112</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt113</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt114</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt115</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt116</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt117</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt118</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt119</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt12</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt120</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt121</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt122</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt123</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt124</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt125</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt126</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt127</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt128</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt129</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt13</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt130</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt131</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt132</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt133</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt134</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt135</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt136</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt137</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt138</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt139</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt14</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt140</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt141</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt142</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt143</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt144</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt145</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt146</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt147</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt148</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt149</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt15</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt150</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt151</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt152</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt153</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt154</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt155</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt156</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt157</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt158</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt159</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt16</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt160</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt161</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt162</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt163</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt164</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt165</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt166</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt167</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt168</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt169</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt17</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt170</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt171</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt172</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt173</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt174</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt175</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt176</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt177</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt178</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt179</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt18</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt180</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt181</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt182</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt183</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt184</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt185</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt186</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt187</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt188</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt189</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt19</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt190</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt191</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt192</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt193</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt194</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt195</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt196</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt197</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt198</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt199</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt2</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt20</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt200</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt201</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt202</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt203</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt204</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt205</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt206</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt207</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt208</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt209</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt21</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt210</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt211</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt212</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt213</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt214</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt215</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt216</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt217</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt218</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt219</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt22</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt220</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt221</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt222</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt223</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt224</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt225</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt226</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt227</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt228</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt229</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt23</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt230</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt231</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt232</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt233</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt234</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt235</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt236</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt237</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt238</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt239</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt24</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt240</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt241</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt242</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt243</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt244</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt245</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt246</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt247</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt248</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt249</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt25</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt250</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt251</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt252</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt253</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt254</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt255</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt256</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt257</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt258</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt259</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt26</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt260</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt261</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt262</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt263</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt264</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt265</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt266</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt267</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt268</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt269</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt27</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt270</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt271</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt272</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt273</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt274</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt275</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt276</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt277</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt278</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt279</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt28</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt280</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt281</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt282</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt283</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt284</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt285</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt286</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt287</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt288</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt289</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt29</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt290</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt291</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt292</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt293</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt294</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt295</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt296</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt297</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt298</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt299</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt3</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt30</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt300</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt301</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt302</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt303</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt304</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt305</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt306</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt307</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt308</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt309</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt31</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt310</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt311</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt312</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt313</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt314</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt315</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt316</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt317</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt318</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt319</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt32</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt320</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt321</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt322</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt323</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt324</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt325</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt326</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt327</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt328</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt329</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt33</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt330</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt331</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt332</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt333</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt334</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt335</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt336</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt337</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt338</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt339</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt34</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt340</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt341</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt342</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt343</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt344</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt345</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt346</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt347</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt348</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt349</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt35</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt350</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt351</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt352</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt353</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt354</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt355</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt356</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt357</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt358</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt359</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt36</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt360</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt361</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt362</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt363</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt364</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt365</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt366</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt367</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt368</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt369</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt37</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt370</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt371</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt372</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt373</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt374</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt375</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt376</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt377</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt378</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt379</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt38</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt380</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt381</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt382</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt383</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt384</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt385</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt386</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt387</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt388</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt389</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt39</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt390</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt391</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt392</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt393</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt394</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt395</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt396</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt397</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt398</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt399</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt4</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt40</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt400</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt401</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt402</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt403</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt404</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt405</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt406</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt407</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt408</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt409</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt41</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt410</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt411</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt412</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt413</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt414</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt415</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt416</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt417</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt418</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt419</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt42</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt420</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt421</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt422</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt423</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt424</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt425</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt426</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt427</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt428</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt429</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt43</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt430</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt431</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt432</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt433</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt434</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt435</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt436</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt437</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt438</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt439</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt44</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt440</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt441</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt442</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt443</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt444</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt445</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt446</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt449</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt45</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt46</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt47</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt48</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt49</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt5</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt50</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt51</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt52</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt53</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt54</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt55</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt56</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt57</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt58</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt59</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt6</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt60</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt61</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt62</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt63</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt64</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt65</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt66</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt67</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt68</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt69</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt7</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt70</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt71</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt72</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt73</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt74</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt75</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt76</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt77</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt78</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt79</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt8</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt80</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt81</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt82</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt83</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt84</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt85</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt86</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt87</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt88</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt89</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt9</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt90</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt91</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt92</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt93</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt94</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt95</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt96</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt97</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt98</str>     <str name="type">tlong</str>   </lst>   <lst>     <str name="name">newTestFieldInt99</str>     <str name="type">tlong</str>   </lst> </arr> </response> 

Stack Trace:
java.lang.AssertionError: QUERY FAILED: xpath=/response/arr[@name='fields']/lst/str[@name='name'][.='newTestFieldInt447']  request=/schema/fields?wt=xml  response=<?xml version="1.0" encoding="UTF-8"?>
<response>

<lst name="responseHeader">
  <int name="status">0</int>
  <int name="QTime">172</int>
</lst>
<arr name="fields">
  <lst>
    <str name="name">_root_</str>
    <str name="type">string</str>
    <bool name="multiValued">false</bool>
    <bool name="stored">true</bool>
    <bool name="indexed">true</bool>
  </lst>
  <lst>
    <str name="name">_version_</str>
    <str name="type">long</str>
    <bool name="stored">true</bool>
    <bool name="indexed">true</bool>
  </lst>
  <lst>
    <str name="name">constantField</str>
    <str name="type">tdouble</str>
  </lst>
  <lst>
    <str name="name">id</str>
    <str name="type">string</str>
    <bool name="required">true</bool>
    <bool name="multiValued">false</bool>
    <bool name="stored">true</bool>
    <bool name="indexed">true</bool>
    <bool name="uniqueKey">true</bool>
  </lst>
  <lst>
    <str name="name">newTestFieldInt0</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt1</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt10</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt100</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt101</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt102</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt103</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt104</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt105</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt106</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt107</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt108</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt109</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt11</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt110</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt111</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt112</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt113</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt114</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt115</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt116</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt117</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt118</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt119</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt12</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt120</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt121</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt122</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt123</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt124</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt125</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt126</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt127</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt128</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt129</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt13</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt130</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt131</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt132</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt133</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt134</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt135</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt136</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt137</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt138</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt139</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt14</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt140</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt141</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt142</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt143</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt144</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt145</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt146</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt147</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt148</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt149</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt15</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt150</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt151</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt152</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt153</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt154</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt155</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt156</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt157</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt158</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt159</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt16</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt160</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt161</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt162</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt163</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt164</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt165</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt166</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt167</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt168</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt169</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt17</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt170</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt171</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt172</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt173</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt174</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt175</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt176</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt177</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt178</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt179</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt18</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt180</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt181</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt182</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt183</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt184</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt185</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt186</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt187</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt188</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt189</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt19</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt190</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt191</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt192</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt193</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt194</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt195</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt196</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt197</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt198</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt199</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt2</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt20</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt200</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt201</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt202</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt203</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt204</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt205</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt206</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt207</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt208</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt209</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt21</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt210</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt211</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt212</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt213</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt214</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt215</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt216</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt217</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt218</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt219</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt22</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt220</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt221</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt222</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt223</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt224</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt225</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt226</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt227</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt228</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt229</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt23</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt230</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt231</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt232</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt233</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt234</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt235</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt236</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt237</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt238</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt239</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt24</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt240</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt241</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt242</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt243</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt244</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt245</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt246</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt247</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt248</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt249</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt25</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt250</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt251</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt252</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt253</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt254</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt255</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt256</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt257</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt258</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt259</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt26</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt260</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt261</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt262</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt263</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt264</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt265</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt266</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt267</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt268</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt269</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt27</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt270</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt271</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt272</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt273</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt274</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt275</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt276</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt277</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt278</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt279</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt28</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt280</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt281</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt282</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt283</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt284</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt285</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt286</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt287</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt288</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt289</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt29</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt290</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt291</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt292</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt293</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt294</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt295</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt296</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt297</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt298</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt299</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt3</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt30</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt300</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt301</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt302</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt303</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt304</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt305</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt306</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt307</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt308</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt309</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt31</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt310</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt311</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt312</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt313</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt314</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt315</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt316</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt317</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt318</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt319</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt32</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt320</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt321</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt322</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt323</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt324</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt325</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt326</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt327</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt328</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt329</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt33</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt330</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt331</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt332</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt333</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt334</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt335</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt336</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt337</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt338</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt339</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt34</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt340</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt341</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt342</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt343</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt344</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt345</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt346</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt347</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt348</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt349</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt35</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt350</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt351</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt352</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt353</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt354</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt355</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt356</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt357</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt358</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt359</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt36</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt360</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt361</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt362</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt363</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt364</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt365</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt366</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt367</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt368</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt369</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt37</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt370</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt371</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt372</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt373</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt374</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt375</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt376</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt377</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt378</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt379</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt38</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt380</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt381</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt382</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt383</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt384</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt385</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt386</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt387</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt388</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt389</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt39</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt390</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt391</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt392</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt393</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt394</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt395</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt396</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt397</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt398</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt399</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt4</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt40</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt400</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt401</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt402</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt403</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt404</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt405</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt406</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt407</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt408</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt409</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt41</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt410</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt411</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt412</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt413</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt414</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt415</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt416</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt417</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt418</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt419</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt42</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt420</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt421</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt422</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt423</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt424</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt425</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt426</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt427</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt428</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt429</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt43</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt430</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt431</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt432</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt433</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt434</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt435</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt436</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt437</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt438</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt439</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt44</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt440</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt441</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt442</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt443</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt444</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt445</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt446</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt449</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt45</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt46</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt47</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt48</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt49</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt5</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt50</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt51</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt52</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt53</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt54</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt55</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt56</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt57</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt58</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt59</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt6</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt60</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt61</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt62</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt63</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt64</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt65</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt66</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt67</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt68</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt69</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt7</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt70</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt71</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt72</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt73</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt74</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt75</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt76</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt77</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt78</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt79</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt8</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt80</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt81</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt82</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt83</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt84</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt85</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt86</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt87</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt88</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt89</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt9</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt90</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt91</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt92</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt93</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt94</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt95</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt96</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt97</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt98</str>
    <str name="type">tlong</str>
  </lst>
  <lst>
    <str name="name">newTestFieldInt99</str>
    <str name="type">tlong</str>
  </lst>
</arr>
</response>

	at __randomizedtesting.SeedInfo.seed([3F443D2AA80900AD:B71002F006F56D55]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.solr.schema.TestCloudSchemaless.test(TestCloudSchemaless.java:149)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:964)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:939)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$

[...truncated too long message...]

in-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-authn-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-authz-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-changelog-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-collective-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-event-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-exception-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-journal-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-normalization-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-operational-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-referral-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-schema-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-subtree-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-trigger-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-jdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-jdbm1-2.0.0-M2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-kerberos-codec-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-ldif-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-mavibot-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-protocol-kerberos-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-protocol-ldap-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-protocol-shared-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-xdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/api-all-1.0.0-M20.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/bcprov-jdk15-1.45.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/cglib-nodep-2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/commons-collections-3.2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/commons-math3-3.4.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/easymock-3.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/ehcache-core-2.4.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/hadoop-common-2.6.0-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/hadoop-hdfs-2.6.0-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/hadoop-minikdc-2.6.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jackson-annotations-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jackson-databind-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jcl-over-slf4j-1.7.7.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jersey-core-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jersey-server-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jetty-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jetty-util-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/mina-core-2.0.0-M5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/objenesis-1.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/analysis/icu/lucene-analyzers-icu-5.5.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-analysis-extras/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/contrib/analysis-extras/lib/icu4j-54.1.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-launcher.jar:/x1/jenkins/.ant/lib/ivy-2.3.0.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-antlr.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-regexp.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-resolver.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-net.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jmf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-oro.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-xalan2.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-logging.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jsch.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit4.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-javamail.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jai.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jdepend.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bsf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-swing.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-netrexx.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bcel.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-testutil.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-log4j.jar:/x1/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.7/lib/tools.jar:/x1/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.3.1.jar com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J0-20151130_075516_857.events @/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J0-20151130_075516_857.suites -stdin
   [junit4] ERROR: JVM J0 ended with an exception: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1503)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:130)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:963)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:960)
   [junit4] 	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4] 	at java.lang.Thread.run(Thread.java:745)

BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: At least one slave process threw an exception, first: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.

Total time: 206 minutes 48 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1031 - Still Failing

[Apache Jenkins Server <je...@builds.apache.org>]

Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1031/

1 tests failed.
FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=177356, name=collection0, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=177356, name=collection0, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:37905: collection already exists: awholynewstresscollection_collection0_0
	at __randomizedtesting.SeedInfo.seed([EA00A48BF6BCDC00]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:574)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:240)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:229)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1573)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1594)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:876)




Build Log:
[...truncated 11392 lines...]
   [junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_EA00A48BF6BCDC00-001/init-core-data-001
   [junit4]   2> 2066769 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[EA00A48BF6BCDC00]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (false)
   [junit4]   2> 2066769 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[EA00A48BF6BCDC00]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   2> 2066774 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 2066775 INFO  (Thread-170089) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 2066775 INFO  (Thread-170089) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 2066875 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.ZkTestServer start zk server on port:42201
   [junit4]   2> 2066875 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2066878 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2066881 INFO  (zkCallback-1839-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1cbf3146 name:ZooKeeperConnection Watcher:127.0.0.1:42201 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2066881 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2066881 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2066881 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 2066895 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2066900 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2066906 INFO  (zkCallback-1840-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@62e4ded3 name:ZooKeeperConnection Watcher:127.0.0.1:42201/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2066907 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2066907 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2066907 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 2066909 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 2066910 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 2066911 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 2066913 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 2066913 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 2066916 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 2066916 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 2066918 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 2066918 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 2066920 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 2066920 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 2066922 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 2066922 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 2066924 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 2066924 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 2066926 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 2066926 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 2066930 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 2066931 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 2066933 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 2066933 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 2066935 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 2066935 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 2066941 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 2066941 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 2066946 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2066965 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2066975 INFO  (zkCallback-1841-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@691a793a name:ZooKeeperConnection Watcher:127.0.0.1:42201/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2066975 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2066975 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2067540 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_EA00A48BF6BCDC00-001/control-001/cores/collection1
   [junit4]   2> 2067545 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 2067550 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@27697030{/,null,AVAILABLE}
   [junit4]   2> 2067580 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.e.j.s.ServerConnector Started ServerConnector@fd7facb{SSL-http/1.1}{127.0.0.1:53156}
   [junit4]   2> 2067580 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.e.j.s.Server Started @2070425ms
   [junit4]   2> 2067580 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostPort=53156, hostContext=/, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_EA00A48BF6BCDC00-001/tempDir-001/control/data, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_EA00A48BF6BCDC00-001/control-001/cores}
   [junit4]   2> 2067581 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@3827de24
   [junit4]   2> 2067581 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_EA00A48BF6BCDC00-001/control-001'
   [junit4]   2> 2067581 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 2067581 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 2067581 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2067598 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2067600 INFO  (zkCallback-1842-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@245149bc name:ZooKeeperConnection Watcher:127.0.0.1:42201/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2067600 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2067600 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2067601 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 2067601 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_EA00A48BF6BCDC00-001/control-001/solr.xml
   [junit4]   2> 2067615 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_EA00A48BF6BCDC00-001/control-001/cores
   [junit4]   2> 2067615 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.CoreContainer New CoreContainer 1580381518
   [junit4]   2> 2067615 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_EA00A48BF6BCDC00-001/control-001]
   [junit4]   2> 2067615 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_EA00A48BF6BCDC00-001/control-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_EA00A48BF6BCDC00-001/control-001/lib
   [junit4]   2> 2067616 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 2067622 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 2067624 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 2067624 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 2067625 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:42201/solr
   [junit4]   2> 2067625 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 2067625 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2067626 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2067632 INFO  (zkCallback-1844-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@65e69d24 name:ZooKeeperConnection Watcher:127.0.0.1:42201 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2067632 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2067632 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2067641 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [n:127.0.0.1:53156_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2067660 INFO  (zkCallback-1845-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@43eb8a24 name:ZooKeeperConnection Watcher:127.0.0.1:42201/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2067660 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [n:127.0.0.1:53156_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2067662 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [n:127.0.0.1:53156_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 2067663 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [n:127.0.0.1:53156_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 2067666 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [n:127.0.0.1:53156_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 2067667 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [n:127.0.0.1:53156_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 2067669 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [n:127.0.0.1:53156_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 2067671 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [n:127.0.0.1:53156_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 2067672 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [n:127.0.0.1:53156_    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 2067673 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [n:127.0.0.1:53156_    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 2067674 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [n:127.0.0.1:53156_    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 2067675 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [n:127.0.0.1:53156_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:53156_
   [junit4]   2> 2067675 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [n:127.0.0.1:53156_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:53156_
   [junit4]   2> 2067676 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [n:127.0.0.1:53156_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 2067683 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [n:127.0.0.1:53156_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 2067685 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [n:127.0.0.1:53156_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 2067685 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [n:127.0.0.1:53156_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94947864418254853-127.0.0.1:53156_-n_0000000000
   [junit4]   2> 2067686 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [n:127.0.0.1:53156_    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:53156_
   [junit4]   2> 2067686 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [n:127.0.0.1:53156_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 2067689 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [n:127.0.0.1:53156_    ] o.a.s.c.Overseer Overseer (id=94947864418254853-127.0.0.1:53156_-n_0000000000) starting
   [junit4]   2> 2067692 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [n:127.0.0.1:53156_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 2067704 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [n:127.0.0.1:53156_    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 2067710 INFO  (OverseerStateUpdate-94947864418254853-127.0.0.1:53156_-n_0000000000) [n:127.0.0.1:53156_    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 2067711 INFO  (OverseerCollectionConfigSetProcessor-94947864418254853-127.0.0.1:53156_-n_0000000000) [n:127.0.0.1:53156_    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 2067711 INFO  (OverseerStateUpdate-94947864418254853-127.0.0.1:53156_-n_0000000000) [n:127.0.0.1:53156_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 2067729 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [n:127.0.0.1:53156_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 2067795 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [n:127.0.0.1:53156_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 2067795 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [n:127.0.0.1:53156_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 2067799 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [n:127.0.0.1:53156_    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_EA00A48BF6BCDC00-001/control-001/cores
   [junit4]   2> 2067799 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [n:127.0.0.1:53156_    ] o.a.s.c.SolrCore Created CoreDescriptor: {absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_EA00A48BF6BCDC00-001/control-001/cores/collection1/, loadOnStartup=true, shard=, dataDir=data/, coreNodeName=, schema=schema.xml, configSetProperties=configsetprops.json, collection=control_collection, name=collection1, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_EA00A48BF6BCDC00-001/control-001/cores/collection1, config=solrconfig.xml, transient=false}
   [junit4]   2> 2067800 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [n:127.0.0.1:53156_    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_EA00A48BF6BCDC00-001/control-001/cores/collection1/
   [junit4]   2> 2067800 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [n:127.0.0.1:53156_    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 2067808 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 2067808 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 2067811 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 2067815 INFO  (OverseerStateUpdate-94947864418254853-127.0.0.1:53156_-n_0000000000) [n:127.0.0.1:53156_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "node_name":"127.0.0.1:53156_",
   [junit4]   2>   "base_url":"https://127.0.0.1:53156",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "shard":null,
   [junit4]   2>   "state":"down",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 2067816 INFO  (OverseerStateUpdate-94947864418254853-127.0.0.1:53156_-n_0000000000) [n:127.0.0.1:53156_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "node_name":"127.0.0.1:53156_",
   [junit4]   2>   "base_url":"https://127.0.0.1:53156",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "shard":null,
   [junit4]   2>   "state":"down",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 2067816 INFO  (OverseerStateUpdate-94947864418254853-127.0.0.1:53156_-n_0000000000) [n:127.0.0.1:53156_    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 2067816 INFO  (OverseerStateUpdate-94947864418254853-127.0.0.1:53156_-n_0000000000) [n:127.0.0.1:53156_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 2067817 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [n:127.0.0.1:53156_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 2067817 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [n:127.0.0.1:53156_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 2067818 INFO  (zkCallback-1845-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2067819 INFO  (zkCallback-1845-thread-2-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2068812 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 2068813 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 2068813 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 2068813 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 2068814 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 2068814 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_EA00A48BF6BCDC00-001/control-001/cores/collection1'
   [junit4]   2> 2068814 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 2068815 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 2068816 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 2068822 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 2068871 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.5.0
   [junit4]   2> 2068909 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 2068925 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 2069374 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 2069381 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 2069382 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 2069404 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 2069407 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 2069409 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 2069410 WARN  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 2069410 WARN  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 2069410 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 2069412 WARN  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 2069412 WARN  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 2069413 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 2069413 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 2069413 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 2069413 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_EA00A48BF6BCDC00-001/control-001/cores/collection1], dataDir=[null]
   [junit4]   2> 2069414 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@783de479
   [junit4]   2> 2069415 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_EA00A48BF6BCDC00-001/control-001/cores/collection1/data
   [junit4]   2> 2069415 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_EA00A48BF6BCDC00-001/control-001/cores/collection1/data/index/
   [junit4]   2> 2069415 WARN  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_EA00A48BF6BCDC00-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 2069415 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_EA00A48BF6BCDC00-001/control-001/cores/collection1/data/index
   [junit4]   2> 2069415 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=9, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=1.853515625, noCFSRatio=0.19011965434550931]
   [junit4]   2> 2069416 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@6293ac8c lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@4d254de3),segFN=segments_1,generation=1}
   [junit4]   2> 2069416 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 2069420 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 2069420 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 2069420 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 2069421 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 2069421 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 2069421 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 2069421 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 2069421 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 2069421 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 2069421 WARN  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
   [junit4]   2> 2069424 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2069426 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2069427 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_EA00A48BF6BCDC00-001/control-001/cores/collection1/data/
   [junit4]   2> 2069427 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_EA00A48BF6BCDC00-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 2069428 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2069430 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2069435 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/file,/schema,standard,/admin/system,/admin/segments,/admin/properties,/update/json/docs,/admin/logging,/config,/replication,/update/csv,/get,/admin/plugins,/admin/luke,/update/json,/dump,/update,/admin/mbeans,/admin/threads,/admin/ping
   [junit4]   2> 2069437 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 2069438 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 2069438 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 2069439 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 2069439 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 2069439 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@65304623
   [junit4]   2> 2069440 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@6293ac8c lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@4d254de3),segFN=segments_1,generation=1}
   [junit4]   2> 2069440 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 2069440 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@105d2567[collection1] main
   [junit4]   2> 2069440 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 2069441 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 2069441 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 2069441 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 2069441 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 2069441 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 2069441 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 2069442 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 2069442 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 2069442 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 2069442 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 2069442 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 2069442 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 2069442 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1519165833487581184
   [junit4]   2> 2069444 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 1.0ms to seed version buckets with highest version 1519165833487581184
   [junit4]   2> 2069444 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 2069444 INFO  (searcherExecutor-3798-thread-1-processing-n:127.0.0.1:53156_ x:collection1 c:control_collection s:shard1 r:core_node1) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@105d2567[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2069445 INFO  (coreLoadExecutor-3797-thread-1-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 2069449 INFO  (coreZkRegister-3791-thread-1-processing-n:127.0.0.1:53156_ x:collection1 c:control_collection s:shard1 r:core_node1) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:https://127.0.0.1:53156 collection:control_collection shard:shard1
   [junit4]   2> 2069450 INFO  (coreZkRegister-3791-thread-1-processing-n:127.0.0.1:53156_ x:collection1 c:control_collection s:shard1 r:core_node1) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 2069453 INFO  (coreZkRegister-3791-thread-1-processing-n:127.0.0.1:53156_ x:collection1 c:control_collection s:shard1 r:core_node1) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94947864418254853-core_node1-n_0000000000
   [junit4]   2> 2069454 INFO  (coreZkRegister-3791-thread-1-processing-n:127.0.0.1:53156_ x:collection1 c:control_collection s:shard1 r:core_node1) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 2069456 INFO  (coreZkRegister-3791-thread-1-processing-n:127.0.0.1:53156_ x:collection1 c:control_collection s:shard1 r:core_node1) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 2069456 INFO  (OverseerStateUpdate-94947864418254853-127.0.0.1:53156_-n_0000000000) [n:127.0.0.1:53156_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 2069456 INFO  (coreZkRegister-3791-thread-1-processing-n:127.0.0.1:53156_ x:collection1 c:control_collection s:shard1 r:core_node1) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 2069457 INFO  (coreZkRegister-3791-thread-1-processing-n:127.0.0.1:53156_ x:collection1 c:control_collection s:shard1 r:core_node1) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:53156/collection1/
   [junit4]   2> 2069457 INFO  (coreZkRegister-3791-thread-1-processing-n:127.0.0.1:53156_ x:collection1 c:control_collection s:shard1 r:core_node1) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 2069457 INFO  (coreZkRegister-3791-thread-1-processing-n:127.0.0.1:53156_ x:collection1 c:control_collection s:shard1 r:core_node1) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy https://127.0.0.1:53156/collection1/ has no replicas
   [junit4]   2> 2069458 INFO  (zkCallback-1845-thread-3-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2069458 INFO  (zkCallback-1845-thread-2-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2069459 INFO  (coreZkRegister-3791-thread-1-processing-n:127.0.0.1:53156_ x:collection1 c:control_collection s:shard1 r:core_node1) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 2069464 INFO  (coreZkRegister-3791-thread-1-processing-n:127.0.0.1:53156_ x:collection1 c:control_collection s:shard1 r:core_node1) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 2069466 INFO  (coreZkRegister-3791-thread-1-processing-n:127.0.0.1:53156_ x:collection1 c:control_collection s:shard1 r:core_node1) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:53156/collection1/ shard1
   [junit4]   2> 2069467 INFO  (OverseerStateUpdate-94947864418254853-127.0.0.1:53156_-n_0000000000) [n:127.0.0.1:53156_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"https://127.0.0.1:53156",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 2
   [junit4]   2> 2069569 INFO  (zkCallback-1845-thread-3-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2069569 INFO  (zkCallback-1845-thread-2-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2069618 INFO  (coreZkRegister-3791-thread-1-processing-n:127.0.0.1:53156_ x:collection1 c:control_collection s:shard1 r:core_node1) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are https://127.0.0.1:53156/collection1/ and leader is https://127.0.0.1:53156/collection1/
   [junit4]   2> 2069618 INFO  (coreZkRegister-3791-thread-1-processing-n:127.0.0.1:53156_ x:collection1 c:control_collection s:shard1 r:core_node1) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:53156
   [junit4]   2> 2069618 INFO  (coreZkRegister-3791-thread-1-processing-n:127.0.0.1:53156_ x:collection1 c:control_collection s:shard1 r:core_node1) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 2069618 INFO  (coreZkRegister-3791-thread-1-processing-n:127.0.0.1:53156_ x:collection1 c:control_collection s:shard1 r:core_node1) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 2069618 INFO  (coreZkRegister-3791-thread-1-processing-n:127.0.0.1:53156_ x:collection1 c:control_collection s:shard1 r:core_node1) [n:127.0.0.1:53156_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 2069620 INFO  (OverseerStateUpdate-94947864418254853-127.0.0.1:53156_-n_0000000000) [n:127.0.0.1:53156_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "node_name":"127.0.0.1:53156_",
   [junit4]   2>   "base_url":"https://127.0.0.1:53156",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "operation":"state"} current state version: 3
   [junit4]   2> 2069621 INFO  (OverseerStateUpdate-94947864418254853-127.0.0.1:53156_-n_0000000000) [n:127.0.0.1:53156_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "node_name":"127.0.0.1:53156_",
   [junit4]   2>   "base_url":"https://127.0.0.1:53156",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 2069738 INFO  (zkCallback-1845-thread-2-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2069738 INFO  (zkCallback-1845-thread-3-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2069843 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2069860 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2069865 INFO  (zkCallback-1847-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7e568b91 name:ZooKeeperConnection Watcher:127.0.0.1:42201/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2069865 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2069865 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2069866 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 2069867 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 2070535 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_EA00A48BF6BCDC00-001/shard-1-001/cores/collection1
   [junit4]   2> 2070537 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_EA00A48BF6BCDC00-001/shard-1-001
   [junit4]   2> 2070548 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 2070588 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@764c07d0{/,null,AVAILABLE}
   [junit4]   2> 2070598 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.e.j.s.ServerConnector Started ServerConnector@2ec7f960{SSL-http/1.1}{127.0.0.1:36896}
   [junit4]   2> 2070598 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.e.j.s.Server Started @2073443ms
   [junit4]   2> 2070598 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solrconfig=solrconfig.xml, hostPort=36896, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_EA00A48BF6BCDC00-001/shard-1-001/cores, hostContext=/, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_EA00A48BF6BCDC00-001/tempDir-001/jetty1}
   [junit4]   2> 2070598 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@3827de24
   [junit4]   2> 2070598 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_EA00A48BF6BCDC00-001/shard-1-001'
   [junit4]   2> 2070599 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 2070599 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 2070599 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2070600 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2070601 INFO  (zkCallback-1848-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@14fd4fa2 name:ZooKeeperConnection Watcher:127.0.0.1:42201/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2070601 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2070601 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2070602 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 2070602 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_EA00A48BF6BCDC00-001/shard-1-001/solr.xml
   [junit4]   2> 2070621 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_EA00A48BF6BCDC00-001/shard-1-001/cores
   [junit4]   2> 2070621 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.CoreContainer New CoreContainer 941909283
   [junit4]   2> 2070621 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_EA00A48BF6BCDC00-001/shard-1-001]
   [junit4]   2> 2070621 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_EA00A48BF6BCDC00-001/shard-1-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_EA00A48BF6BCDC00-001/shard-1-001/lib
   [junit4]   2> 2070621 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 2070634 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 2070636 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 2070637 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 2070637 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:42201/solr
   [junit4]   2> 2070637 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 2070637 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2070645 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2070645 INFO  (zkCallback-1850-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1f2d958 name:ZooKeeperConnection Watcher:127.0.0.1:42201 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2070645 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2070646 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2070653 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [n:127.0.0.1:36896_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2070654 INFO  (zkCallback-1851-thread-1-processing-n:127.0.0.1:36896_) [n:127.0.0.1:36896_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@aad92d7 name:ZooKeeperConnection Watcher:127.0.0.1:42201/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2070654 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [n:127.0.0.1:36896_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2070669 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [n:127.0.0.1:36896_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 2071675 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [n:127.0.0.1:36896_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:36896_
   [junit4]   2> 2071675 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [n:127.0.0.1:36896_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:36896_
   [junit4]   2> 2071676 INFO  (zkCallback-1847-thread-1) [    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2071676 INFO  (zkCallback-1845-thread-3-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2071676 INFO  (zkCallback-1845-thread-2-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2071676 INFO  (zkCallback-1851-thread-1-processing-n:127.0.0.1:36896_) [n:127.0.0.1:36896_    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2071679 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [n:127.0.0.1:36896_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 2071680 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [n:127.0.0.1:36896_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94947864418254857-127.0.0.1:36896_-n_0000000001
   [junit4]   2> 2071680 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [n:127.0.0.1:36896_    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/94947864418254853-127.0.0.1:53156_-n_0000000000 to know if I could be the leader
   [junit4]   2> 2071835 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [n:127.0.0.1:36896_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 2071835 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [n:127.0.0.1:36896_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 2071836 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [n:127.0.0.1:36896_    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_EA00A48BF6BCDC00-001/shard-1-001/cores
   [junit4]   2> 2071837 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [n:127.0.0.1:36896_    ] o.a.s.c.SolrCore Created CoreDescriptor: {collection=collection1, coreNodeName=, name=collection1, dataDir=data/, shard=, loadOnStartup=true, config=solrconfig.xml, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_EA00A48BF6BCDC00-001/shard-1-001/cores/collection1, transient=false, configSetProperties=configsetprops.json, schema=schema.xml, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_EA00A48BF6BCDC00-001/shard-1-001/cores/collection1/}
   [junit4]   2> 2071837 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [n:127.0.0.1:36896_    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_EA00A48BF6BCDC00-001/shard-1-001/cores/collection1/
   [junit4]   2> 2071838 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [n:127.0.0.1:36896_    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 2071839 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [n:127.0.0.1:36896_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 2071839 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[EA00A48BF6BCDC00]) [n:127.0.0.1:36896_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 2071839 INFO  (coreLoadExecutor-3808-thread-1-processing-n:127.0.0.1:36896_) [n:127.0.0.1:36896_ c:collection1   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 2071839 INFO  (coreLoadExecutor-3808-thread-1-processing-n:127.0.0.1:36896_) [n:127.0.0.1:36896_ c:collection1   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 2071841 INFO  (OverseerStateUpdate-94947864418254853-127.0.0.1:53156_-n_0000000000) [n:127.0.0.1:53156_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "node_name":"127.0.0.1:36896_",
   [junit4]   2>   "base_url":"https://127.0.0.1:36896",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "shard":null,
   [junit4]   2>   "state":"down",
   [junit4]   2>   "operation":"state"} current state version: 4
   [junit4]   2> 2071841 INFO  (OverseerStateUpdate-94947864418254853-127.0.0.1:53156_-n_0000000000) [n:127.0.0.1:53156_    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "node_name":"127.0.0.1:36896_",
   [junit4]   2>   "base_url":"https://127.0.0.1:36896",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "shard":null,
   [junit4]   2>   "state":"down",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 2071841 INFO  (coreLoadExecutor-3808-thread-1-processing-n:127.0.0.1:36896_) [n:127.0.0.1:36896_ c:collection1   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 2071841 INFO  (OverseerStateUpdate-94947864418254853-127.0.0.1:53156_-n_0000000000) [n:127.0.0.1:53156_    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 2071842 INFO  (OverseerStateUpdate-94947864418254853-127.0.0.1:53156_-n_0000000000) [n:127.0.0.1:53156_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 2071844 INFO  (zkCallback-1847-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 2071844 INFO  (zkCallback-1851-thread-1-processing-n:127.0.0.1:36896_) [n:127.0.0.1:36896_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 2071844 INFO  (zkCallback-1845-thread-2-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 2071844 INFO  (zkCallback-1845-thread-3-processing-n:127.0.0.1:53156_) [n:127.0.0.1:53156_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 2072843 INFO  (coreLoadExecutor-3808-thread-1-processing-n:127.0.0.1:36896_) [n:127.0.0.1:36896_ c:collection1   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 2072843 INFO  (coreLoadExecutor-3808-thread-1-processing-n:127.0.0.1:36896_) [n:127.0.0.1:36896_ c:collection1   x:collection1] o.a.s.c.ZkController Check for collection zkNode:collection1
   [junit4]   2> 2072843 INFO  (coreLoadExecutor-3808-thread-1-processing-n:127.0.0.1:36896_) [n:127.0.0.1:36896_ c:collection1   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 2072843 INFO  (coreLoadExecutor-3808-thread-1-processing-n:127.0.0.1:36896_) [n:127.0.0.1:36896_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/collection1
   [junit4]   2> 2072844 INFO  (coreLoadExecutor-3808-thread-1-processing-n:127.0.0.1:36896_) [n:127.0.0.1:36896_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 2072844 INFO  (coreLoadExecutor-3808-thread-1-processing-n:127.0.0.1:36896_) [n:127.0.0.1:36896_ c:collection1   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_EA00A48BF6BCDC00-001/shard-1-001/cores/collection1'
   [junit4]   2> 2072844 INFO  (coreLoadExecutor-3808-thread-1-processing-n:127.0.0.1:36896_) [n:127.0.0.1:36896_ c:collection1   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 2072844 INFO  (coreLoadExecutor-3808-thread-1-processing-n:127.0.0.1:36896_) [n:127.0.0.1:36896_ c:collection1   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 2072846 INFO  (coreLoadExecutor-3808-thread-1-processing-n:127.0.0.1:36896_) [n:127.0.0.1:36896_ c:collection1   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 2072852 INFO  (coreLoadExecutor-3808-thread-1-processing-n:127.0.0.1:36896_) [n:127.0.0.1:36896_ c:collection1   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 2072866 INFO  (coreLoadExecutor-3808-thread-1-processing-n:127.0.0.1:36896_) [n:127.0.0.1:36896_ c:collection1   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.5.0
   [junit4]   2> 2072945 INFO  (coreLoadExecutor-3808-thread-1-processing-n:127.0.0.1:36896_) [n:127.0.0.1:36896_ c:collection1   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 2072957 INFO  (coreLoadExecutor-3808-thread-1-processing-n:127.0.0.1:36896_) [n:127.0.0.1:36896_ c:collection1   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 2073423 INFO  (coreLoadExecutor-3808-thread-1-processing-n:127.0.0.1:36896_) [n:127.0.0.1:36896_ c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 2073437 INFO  (coreLoadExecutor-3808-thread-1-processing-n:127.0.0.1:36896_) [n:127.0.0.1:36896_ c:collection1   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 2073439 INFO  (coreLoadExecutor-3808-thread-1-processing-n:127.0.0.1:36896_) [n:127.0.0.1:36896_ c:collection1   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 2073461 INFO  (coreLoadExecutor-3808-thread-1-processing-n:127.0.0.1:36896_) [n:127.0.0.1:36896_ c:collection1   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 2073467 INFO  (coreLoadExecutor-3808-thread-1-processing-n:127.0.0.1:36896_) [n:127.0.0.1:36896_ c:collection1   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 2073471 INFO  (coreLoadExecutor-3808-thread-1-processing-n:127.0.0.1:36896_) [n:127.0.0.1:36896_ c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 2073473 WARN  (coreLoadExecutor-3808-thread-1-processing-n:127.0.0.1:36896_) [n:127.0.0.1:36896_ c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 2073474 WARN  (coreLoadExecutor-3808-thread-1-processing-n:127.0.0.1:36896_) [n:127.0.0.1:36896_ c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 2073474 INFO  (coreLoadExecutor-3808-thread-1-processing-n:127.0.0.1:36896_) [n:127.0.0.1:36896_ c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 2073476 WARN  (coreLoadExecutor-3808-thread-1-processing-n:127.0.0.1:36896_) [n:127.0.0.1:36896_ c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 2073476 WARN  (coreLoadExecutor-3808-thread-1-processing-n:127.0.0.1:36896_) [n:127.0.0.1:36896_ c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 2073477 INFO  (coreLoadExecutor-3808-thread-1-processing-n:127.0.0.1:36896_) [n:127.0.0.1:36896_ c:collection1   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 2073477 INFO  (coreLoadExecutor-3808-thread-1-processing-n:127.0.0.1:36896_) [n:127.0.0.1:36896_ c:collection1   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection collection1
   [junit4]   2> 2073477 INFO  (coreLoadExecutor-3808-thread-1-processing-n:127.0.0.1:36896_) [n:127.0.0.1:36896_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 2073477 INFO  (coreLoadExecutor-3808-thread-1-processing-n:127.0.0.1:36896_) [n:127.0.0.1:36896_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_EA00A48BF6BCDC00-001/shard-1-001/cores/collection1], dataDir=[null]
   [junit4]   2> 2073477 INFO  (coreLoadExecutor-3808-thread-1-processing-n:127.0.0.1:36896_) [n:127.0.0.1:36896_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@783de479
   [junit4]   2> 2073478 INFO  (coreLoadExecutor-3808-thread-1-processing-n:127.0.0.1:36896_) [n:127.0.0.1:36896_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_EA00A48BF6BCDC00-001/shard-1-001/cores/collection1/data
   [junit4]   2> 2073478 INFO  (coreLoadExecutor-3808-thread-1-processing-n:127.0.0.1:36896_) [n:127.0.0.1:36896_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_EA00A48BF6BCDC00-001/shard-1-001/cores/collection1/data/index/
   [junit4]   2> 2073478 WARN  (coreLoadExecutor-3808-thread-1-processing-n:127.0.0.1:36896_) [n:127.0.0.1:36896_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_EA00A48BF6BCDC00-001/shard-1-001/cores/collection1/d

[...truncated too long message...]

lrj.SolrClient.request(SolrClient.java:1220)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1573)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1594)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:876)Throwable #5: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=177360, name=collection4, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    > Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:37905: collection already exists: awholynewstresscollection_collection4_0
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([EA00A48BF6BCDC00]:0)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:574)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:240)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:229)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1573)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1594)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:876)Throwable #6: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=177359, name=collection3, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    > Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:36361: collection already exists: awholynewstresscollection_collection3_0
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([EA00A48BF6BCDC00]:0)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:574)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:240)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:229)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1573)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1594)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:876)Throwable #7: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=177358, name=collection2, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    > Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:36361: Could not find collection : awholynewstresscollection_collection2_0
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([EA00A48BF6BCDC00]:0)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:574)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:240)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:229)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:883)
   [junit4]   2> 2854653 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[EA00A48BF6BCDC00]-worker) [n:127.0.0.1:36361_ c:halfdeletedcollection2 s:shard2 r:core_node9 x:halfdeletedcollection2_shard2_replica2] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_EA00A48BF6BCDC00-001
   [junit4]   2> Nov 29, 2015 9:43:42 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=CheapBastard, sim=RandomSimilarityProvider(queryNorm=false,coord=yes): {}, locale=ar_LY, timezone=Europe/Zurich
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_80 (64-bit)/cpus=4,threads=1,free=53993784,total=429916160
   [junit4]   2> NOTE: All tests run in this JVM: [ClusterStateUpdateTest, TestManagedSchemaFieldTypeResource, TestLeaderInitiatedRecoveryThread, TestAddFieldRealTimeGet, TestSolrCoreProperties, RollingRestartTest, TestSolrConfigHandlerCloud, IndexBasedSpellCheckerTest, TestDocumentBuilder, TestQueryTypes, TestInitQParser, UUIDUpdateProcessorFallbackTest, TimeZoneUtilsTest, TestFreeTextSuggestions, ChaosMonkeyNothingIsSafeTest, DataDrivenBlockJoinTest, DistributedTermsComponentTest, ZkSolrClientTest, TestExpandComponent, TestRestManager, NumericFieldsTest, BigEndianAscendingWordDeserializerTest, TestClusterStateMutator, SpellPossibilityIteratorTest, SolrIndexSplitterTest, OpenExchangeRatesOrgProviderTest, TestSystemIdResolver, TestOverriddenPrefixQueryForCustomFieldType, TestClassNameShortening, SolrPluginUtilsTest, TestSerializedLuceneMatchVersion, ShardSplitTest, SuggestComponentTest, TestStandardQParsers, TestManagedSchemaFieldResource, TestSolrQueryParserDefaultOperatorResource, HdfsBasicDistributedZkTest, SpatialRPTFieldTypeTest, DocValuesTest, TestConfigSetsAPIZkFailure, TestRuleBasedAuthorizationPlugin, BitVectorTest, TestSchemaResource, HdfsRecoverLeaseTest, TestCopyFieldCollectionResource, WordBreakSolrSpellCheckerTest, TestFiltering, FullHLLTest, TestCodecSupport, TestRequestForwarding, BadCopyFieldTest, SpellCheckCollatorTest, MigrateRouteKeyTest, TestJettySolrRunner, TestBM25SimilarityFactory, FastVectorHighlighterTest, SecurityConfHandlerTest, SpellCheckComponentTest, SpellingQueryConverterTest, MultiTermTest, SpatialFilterTest, UpdateRequestProcessorFactoryTest, SimpleMLTQParserTest, SolrInfoMBeanTest, TestExactStatsCache, DistanceFunctionTest, SimplePostToolTest, DeleteReplicaTest, TestSortingResponseWriter, TestMiniSolrCloudClusterSSL, TestManagedStopFilterFactory, TestPartialUpdateDeduplication, CollectionsAPIDistributedZkTest]
   [junit4] Completed [406/551] on J1 in 788.27s, 1 test, 1 error <<< FAILURES!

[...truncated 474 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 551 suites (7 ignored), 2177 tests, 1 error, 32 ignored (18 assumptions) [seed: EA00A48BF6BCDC00]

Total time: 211 minutes 55 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1030 - Still Failing

[Apache Jenkins Server <je...@builds.apache.org>]

Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1030/

1 tests failed.
FAILED:  org.apache.solr.TestDistributedSearch.test

Error Message:


Stack Trace:
java.lang.NullPointerException
	at __randomizedtesting.SeedInfo.seed([DA3F7411FFE8837F:526B4BCB5114EE87]:0)
	at org.apache.solr.TestDistributedSearch.test(TestDistributedSearch.java:568)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsRepeatStatement.callStatement(BaseDistributedSearchTestCase.java:986)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:939)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 11014 lines...]
   [junit4] Suite: org.apache.solr.TestDistributedSearch
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.TestDistributedSearch_DA3F7411FFE8837F-001/init-core-data-001
   [junit4]   2> 2164383 INFO  (SUITE-TestDistributedSearch-seed#[DA3F7411FFE8837F]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false)
   [junit4]   2> 2164383 INFO  (SUITE-TestDistributedSearch-seed#[DA3F7411FFE8837F]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   2> 2164666 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.TestDistributedSearch_DA3F7411FFE8837F-001/tempDir-001/control/cores/collection1
   [junit4]   2> 2164667 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 2164669 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@174bee47{/,null,AVAILABLE}
   [junit4]   2> 2164669 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    ] o.e.j.s.ServerConnector Started ServerConnector@44e1e24c{HTTP/1.1}{127.0.0.1:47605}
   [junit4]   2> 2164669 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    ] o.e.j.s.Server Started @2167062ms
   [junit4]   2> 2164669 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.TestDistributedSearch_DA3F7411FFE8837F-001/tempDir-001/control/cores, hostPort=47605, hostContext=/}
   [junit4]   2> 2164669 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@7b3cb2c6
   [junit4]   2> 2164669 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.TestDistributedSearch_DA3F7411FFE8837F-001/tempDir-001/control'
   [junit4]   2> 2164670 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 2164670 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 2164670 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.TestDistributedSearch_DA3F7411FFE8837F-001/tempDir-001/control/solr.xml
   [junit4]   2> 2164684 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.TestDistributedSearch_DA3F7411FFE8837F-001/tempDir-001/control/cores
   [junit4]   2> 2164684 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    ] o.a.s.c.CoreContainer New CoreContainer 1951022864
   [junit4]   2> 2164684 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.TestDistributedSearch_DA3F7411FFE8837F-001/tempDir-001/control]
   [junit4]   2> 2164684 WARN  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.TestDistributedSearch_DA3F7411FFE8837F-001/tempDir-001/control/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.TestDistributedSearch_DA3F7411FFE8837F-001/tempDir-001/control/lib
   [junit4]   2> 2164684 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 2164685 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 2164686 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 2164686 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 2164686 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 2164686 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 2164687 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.TestDistributedSearch_DA3F7411FFE8837F-001/tempDir-001/control/cores
   [junit4]   2> 2164687 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    ] o.a.s.c.SolrCore Created CoreDescriptor: {dataDir=data/, configSetProperties=configsetprops.json, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.TestDistributedSearch_DA3F7411FFE8837F-001/tempDir-001/control/cores/collection1, loadOnStartup=true, transient=false, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.TestDistributedSearch_DA3F7411FFE8837F-001/tempDir-001/control/cores/collection1/, config=solrconfig.xml, configSet=collection1, schema=schema.xml, name=collection1}
   [junit4]   2> 2164687 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.TestDistributedSearch_DA3F7411FFE8837F-001/tempDir-001/control/cores/collection1/
   [junit4]   2> 2164687 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 2164688 INFO  (coreLoadExecutor-4498-thread-1) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1'
   [junit4]   2> 2164688 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 2164688 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 2164688 INFO  (coreLoadExecutor-4498-thread-1) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 2164688 INFO  (coreLoadExecutor-4498-thread-1) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 2164688 INFO  (coreLoadExecutor-4498-thread-1) [    ] o.a.s.c.SolrResourceLoader Adding 'file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/.svn/' to classloader
   [junit4]   2> 2164688 INFO  (coreLoadExecutor-4498-thread-1) [    ] o.a.s.c.SolrResourceLoader Adding 'file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/classes/' to classloader
   [junit4]   2> 2164688 INFO  (coreLoadExecutor-4498-thread-1) [    ] o.a.s.c.SolrResourceLoader Adding 'file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/README' to classloader
   [junit4]   2> 2164715 INFO  (coreLoadExecutor-4498-thread-1) [    ] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 2164734 INFO  (coreLoadExecutor-4498-thread-1) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.5.0
   [junit4]   2> 2164777 INFO  (coreLoadExecutor-4498-thread-1) [    ] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 2164784 INFO  (coreLoadExecutor-4498-thread-1) [    ] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 2165020 INFO  (coreLoadExecutor-4498-thread-1) [    ] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 2165027 INFO  (coreLoadExecutor-4498-thread-1) [    ] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 2165029 INFO  (coreLoadExecutor-4498-thread-1) [    ] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 2165033 INFO  (coreLoadExecutor-4498-thread-1) [    ] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 2165036 INFO  (coreLoadExecutor-4498-thread-1) [    ] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 2165037 INFO  (coreLoadExecutor-4498-thread-1) [    ] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 2165038 WARN  (coreLoadExecutor-4498-thread-1) [    ] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 2165038 WARN  (coreLoadExecutor-4498-thread-1) [    ] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 2165038 INFO  (coreLoadExecutor-4498-thread-1) [    ] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 2165038 WARN  (coreLoadExecutor-4498-thread-1) [    ] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 2165038 WARN  (coreLoadExecutor-4498-thread-1) [    ] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 2165039 INFO  (coreLoadExecutor-4498-thread-1) [    ] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1'
   [junit4]   2> 2165040 INFO  (coreLoadExecutor-4498-thread-1) [    ] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from configset /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1
   [junit4]   2> 2165040 INFO  (coreLoadExecutor-4498-thread-1) [    x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 2165040 INFO  (coreLoadExecutor-4498-thread-1) [    x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1], dataDir=[null]
   [junit4]   2> 2165040 INFO  (coreLoadExecutor-4498-thread-1) [    x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@19efd160
   [junit4]   2> 2165041 INFO  (coreLoadExecutor-4498-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.TestDistributedSearch_DA3F7411FFE8837F-001/tempDir-001/control/cores/collection1/data
   [junit4]   2> 2165041 INFO  (coreLoadExecutor-4498-thread-1) [    x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.TestDistributedSearch_DA3F7411FFE8837F-001/tempDir-001/control/cores/collection1/data/index/
   [junit4]   2> 2165041 WARN  (coreLoadExecutor-4498-thread-1) [    x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.TestDistributedSearch_DA3F7411FFE8837F-001/tempDir-001/control/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 2165041 INFO  (coreLoadExecutor-4498-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.TestDistributedSearch_DA3F7411FFE8837F-001/tempDir-001/control/cores/collection1/data/index
   [junit4]   2> 2165041 INFO  (coreLoadExecutor-4498-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=27, maxMergeAtOnceExplicit=45, maxMergedSegmentMB=1.17578125, floorSegmentMB=1.2587890625, forceMergeDeletesPctAllowed=25.58422740531904, segmentsPerTier=5.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.532752684939979
   [junit4]   2> 2165043 INFO  (coreLoadExecutor-4498-thread-1) [    x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(SimpleFSDirectory@/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.TestDistributedSearch_DA3F7411FFE8837F-001/index-SimpleFSDirectory-002 lockFactory=org.apache.lucene.store.NativeFSLockFactory@29f68ad9),segFN=segments_1,generation=1}
   [junit4]   2> 2165043 INFO  (coreLoadExecutor-4498-thread-1) [    x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 2165044 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [    x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.TestDistributedSearch_DA3F7411FFE8837F-001/tempDir-001/control/cores/collection1/data/
   [junit4]   2> 2165044 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [    x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.TestDistributedSearch_DA3F7411FFE8837F-001/tempDir-001/control/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 2165047 INFO  (coreLoadExecutor-4498-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 2165047 INFO  (coreLoadExecutor-4498-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 2165048 INFO  (coreLoadExecutor-4498-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe-allfields"
   [junit4]   2> 2165048 INFO  (coreLoadExecutor-4498-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
   [junit4]   2> 2165048 INFO  (coreLoadExecutor-4498-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 2165048 INFO  (coreLoadExecutor-4498-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 2165048 INFO  (coreLoadExecutor-4498-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "uniq-fields"
   [junit4]   2> 2165048 INFO  (coreLoadExecutor-4498-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
   [junit4]   2> 2165048 INFO  (coreLoadExecutor-4498-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 2165048 INFO  (coreLoadExecutor-4498-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 2165048 INFO  (coreLoadExecutor-4498-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 2165048 INFO  (coreLoadExecutor-4498-thread-1) [    x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 2165050 INFO  (coreLoadExecutor-4498-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2165052 INFO  (coreLoadExecutor-4498-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2165053 INFO  (coreLoadExecutor-4498-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2165054 INFO  (coreLoadExecutor-4498-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2165057 INFO  (coreLoadExecutor-4498-thread-1) [    x:collection1] o.a.s.c.PluginBag requestHandler : 'lazy' created with startup=lazy 
   [junit4]   2> 2165059 INFO  (coreLoadExecutor-4498-thread-1) [    x:collection1] o.a.s.c.RequestHandlers Registered paths: /schema,defaults,/get,/config,lazy,mltrh,/replication,tvrh,/debug/dump,/admin/segments,spellCheckCompRH,/update/json,/admin/properties,spellCheckCompRH_Direct,/admin/file,/admin/logging,/terms,/update/json/docs,/search-facet-def,/admin/threads,/search-facet-invariants,/admin/ping,/mlt,standard,/update,/update/csv,/admin/system,mock,spellCheckWithWordbreak,/admin/mbeans,spellCheckWithWordbreak_Direct,/admin/plugins,spellCheckCompRH1,dismax,/admin/luke
   [junit4]   2> 2165060 WARN  (coreLoadExecutor-4498-thread-1) [    x:collection1] o.a.s.c.SolrCore solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
   [junit4]   2> 2165061 INFO  (coreLoadExecutor-4498-thread-1) [    x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 2165061 INFO  (coreLoadExecutor-4498-thread-1) [    x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 2165061 INFO  (coreLoadExecutor-4498-thread-1) [    x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 2165062 INFO  (coreLoadExecutor-4498-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=29, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
   [junit4]   2> 2165063 INFO  (coreLoadExecutor-4498-thread-1) [    x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(SimpleFSDirectory@/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.TestDistributedSearch_DA3F7411FFE8837F-001/index-SimpleFSDirectory-002 lockFactory=org.apache.lucene.store.NativeFSLockFactory@29f68ad9),segFN=segments_1,generation=1}
   [junit4]   2> 2165063 INFO  (coreLoadExecutor-4498-thread-1) [    x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 2165063 INFO  (coreLoadExecutor-4498-thread-1) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@483824f8[collection1] main
   [junit4]   2> 2165063 INFO  (coreLoadExecutor-4498-thread-1) [    x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {storageIO=org.apache.solr.rest.ManagedResourceStorage$InMemoryStorageIO}
   [junit4]   2> 2165063 INFO  (coreLoadExecutor-4498-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using InMemoryStorage
   [junit4]   2> 2165063 INFO  (coreLoadExecutor-4498-thread-1) [    x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 2165063 INFO  (coreLoadExecutor-4498-thread-1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Initializing spell checkers
   [junit4]   2> 2165072 INFO  (coreLoadExecutor-4498-thread-1) [    x:collection1] o.a.s.s.DirectSolrSpellChecker init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
   [junit4]   2> 2165099 INFO  (coreLoadExecutor-4498-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 2165099 INFO  (searcherExecutor-4499-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: default
   [junit4]   2> 2165099 INFO  (searcherExecutor-4499-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: direct
   [junit4]   2> 2165099 INFO  (searcherExecutor-4499-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: wordbreak
   [junit4]   2> 2165099 INFO  (searcherExecutor-4499-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: multipleFields
   [junit4]   2> 2165100 INFO  (searcherExecutor-4499-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: jarowinkler
   [junit4]   2> 2165100 INFO  (searcherExecutor-4499-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: external
   [junit4]   2> 2165100 INFO  (coreLoadExecutor-4498-thread-1) [    x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 2165100 INFO  (searcherExecutor-4499-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: freq
   [junit4]   2> 2165100 INFO  (searcherExecutor-4499-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: fqcn
   [junit4]   2> 2165101 INFO  (searcherExecutor-4499-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: perDict
   [junit4]   2> 2165101 INFO  (searcherExecutor-4499-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@483824f8[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2165445 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.TestDistributedSearch_DA3F7411FFE8837F-001/tempDir-001/shard0/cores/collection1
   [junit4]   2> 2165447 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 2165448 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@2cd230f9{/,null,AVAILABLE}
   [junit4]   2> 2165448 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    ] o.e.j.s.ServerConnector Started ServerConnector@5703ce95{HTTP/1.1}{127.0.0.1:55549}
   [junit4]   2> 2165449 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    ] o.e.j.s.Server Started @2167841ms
   [junit4]   2> 2165449 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostPort=55549, hostContext=/, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.TestDistributedSearch_DA3F7411FFE8837F-001/tempDir-001/shard0/cores}
   [junit4]   2> 2165449 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@7b3cb2c6
   [junit4]   2> 2165449 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.TestDistributedSearch_DA3F7411FFE8837F-001/tempDir-001/shard0'
   [junit4]   2> 2165449 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 2165449 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 2165449 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.TestDistributedSearch_DA3F7411FFE8837F-001/tempDir-001/shard0/solr.xml
   [junit4]   2> 2165472 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.TestDistributedSearch_DA3F7411FFE8837F-001/tempDir-001/shard0/cores
   [junit4]   2> 2165473 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    ] o.a.s.c.CoreContainer New CoreContainer 348463432
   [junit4]   2> 2165473 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.TestDistributedSearch_DA3F7411FFE8837F-001/tempDir-001/shard0]
   [junit4]   2> 2165473 WARN  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.TestDistributedSearch_DA3F7411FFE8837F-001/tempDir-001/shard0/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.TestDistributedSearch_DA3F7411FFE8837F-001/tempDir-001/shard0/lib
   [junit4]   2> 2165473 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 2165474 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 2165474 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 2165475 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 2165475 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 2165475 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 2165475 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.TestDistributedSearch_DA3F7411FFE8837F-001/tempDir-001/shard0/cores
   [junit4]   2> 2165476 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    ] o.a.s.c.SolrCore Created CoreDescriptor: {absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.TestDistributedSearch_DA3F7411FFE8837F-001/tempDir-001/shard0/cores/collection1/, name=collection1, schema=schema.xml, loadOnStartup=true, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.TestDistributedSearch_DA3F7411FFE8837F-001/tempDir-001/shard0/cores/collection1, configSetProperties=configsetprops.json, configSet=collection1, config=solrconfig.xml, dataDir=data/, transient=false}
   [junit4]   2> 2165476 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.TestDistributedSearch_DA3F7411FFE8837F-001/tempDir-001/shard0/cores/collection1/
   [junit4]   2> 2165476 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 2165476 INFO  (coreLoadExecutor-4508-thread-1) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1'
   [junit4]   2> 2165477 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 2165477 INFO  (coreLoadExecutor-4508-thread-1) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 2165477 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 2165477 INFO  (coreLoadExecutor-4508-thread-1) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 2165477 INFO  (coreLoadExecutor-4508-thread-1) [    ] o.a.s.c.SolrResourceLoader Adding 'file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/.svn/' to classloader
   [junit4]   2> 2165477 INFO  (coreLoadExecutor-4508-thread-1) [    ] o.a.s.c.SolrResourceLoader Adding 'file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/classes/' to classloader
   [junit4]   2> 2165477 INFO  (coreLoadExecutor-4508-thread-1) [    ] o.a.s.c.SolrResourceLoader Adding 'file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/README' to classloader
   [junit4]   2> 2165501 INFO  (coreLoadExecutor-4508-thread-1) [    ] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 2165519 INFO  (coreLoadExecutor-4508-thread-1) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.5.0
   [junit4]   2> 2165557 INFO  (coreLoadExecutor-4508-thread-1) [    ] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 2165563 INFO  (coreLoadExecutor-4508-thread-1) [    ] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 2165793 INFO  (coreLoadExecutor-4508-thread-1) [    ] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 2165799 INFO  (coreLoadExecutor-4508-thread-1) [    ] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 2165801 INFO  (coreLoadExecutor-4508-thread-1) [    ] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 2165805 INFO  (coreLoadExecutor-4508-thread-1) [    ] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 2165808 INFO  (coreLoadExecutor-4508-thread-1) [    ] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 2165810 INFO  (coreLoadExecutor-4508-thread-1) [    ] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 2165810 WARN  (coreLoadExecutor-4508-thread-1) [    ] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 2165810 WARN  (coreLoadExecutor-4508-thread-1) [    ] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 2165810 INFO  (coreLoadExecutor-4508-thread-1) [    ] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 2165810 WARN  (coreLoadExecutor-4508-thread-1) [    ] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 2165810 WARN  (coreLoadExecutor-4508-thread-1) [    ] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 2165812 INFO  (coreLoadExecutor-4508-thread-1) [    ] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1'
   [junit4]   2> 2165812 INFO  (coreLoadExecutor-4508-thread-1) [    ] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from configset /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1
   [junit4]   2> 2165812 INFO  (coreLoadExecutor-4508-thread-1) [    x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 2165812 INFO  (coreLoadExecutor-4508-thread-1) [    x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1], dataDir=[null]
   [junit4]   2> 2165812 INFO  (coreLoadExecutor-4508-thread-1) [    x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@19efd160
   [junit4]   2> 2165813 INFO  (coreLoadExecutor-4508-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.TestDistributedSearch_DA3F7411FFE8837F-001/tempDir-001/shard0/cores/collection1/data
   [junit4]   2> 2165813 INFO  (coreLoadExecutor-4508-thread-1) [    x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.TestDistributedSearch_DA3F7411FFE8837F-001/tempDir-001/shard0/cores/collection1/data/index/
   [junit4]   2> 2165813 WARN  (coreLoadExecutor-4508-thread-1) [    x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.TestDistributedSearch_DA3F7411FFE8837F-001/tempDir-001/shard0/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 2165814 INFO  (coreLoadExecutor-4508-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.TestDistributedSearch_DA3F7411FFE8837F-001/tempDir-001/shard0/cores/collection1/data/index
   [junit4]   2> 2165814 INFO  (coreLoadExecutor-4508-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=27, maxMergeAtOnceExplicit=45, maxMergedSegmentMB=1.17578125, floorSegmentMB=1.2587890625, forceMergeDeletesPctAllowed=25.58422740531904, segmentsPerTier=5.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.532752684939979
   [junit4]   2> 2165816 INFO  (coreLoadExecutor-4508-thread-1) [    x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(SimpleFSDirectory@/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.TestDistributedSearch_DA3F7411FFE8837F-001/index-SimpleFSDirectory-004 lockFactory=org.apache.lucene.store.NativeFSLockFactory@29f68ad9),segFN=segments_1,generation=1}
   [junit4]   2> 2165816 INFO  (coreLoadExecutor-4508-thread-1) [    x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 2165816 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [    x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.TestDistributedSearch_DA3F7411FFE8837F-001/tempDir-001/shard0/cores/collection1/data/
   [junit4]   2> 2165817 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [    x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.TestDistributedSearch_DA3F7411FFE8837F-001/tempDir-001/shard0/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 2165821 INFO  (coreLoadExecutor-4508-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 2165821 INFO  (coreLoadExecutor-4508-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 2165821 INFO  (coreLoadExecutor-4508-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe-allfields"
   [junit4]   2> 2165821 INFO  (coreLoadExecutor-4508-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
   [junit4]   2> 2165821 INFO  (coreLoadExecutor-4508-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 2165822 INFO  (coreLoadExecutor-4508-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 2165822 INFO  (coreLoadExecutor-4508-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "uniq-fields"
   [junit4]   2> 2165822 INFO  (coreLoadExecutor-4508-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
   [junit4]   2> 2165822 INFO  (coreLoadExecutor-4508-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 2165822 INFO  (coreLoadExecutor-4508-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 2165822 INFO  (coreLoadExecutor-4508-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 2165822 INFO  (coreLoadExecutor-4508-thread-1) [    x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 2165824 INFO  (coreLoadExecutor-4508-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2165826 INFO  (coreLoadExecutor-4508-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2165827 INFO  (coreLoadExecutor-4508-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2165828 INFO  (coreLoadExecutor-4508-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2165833 INFO  (coreLoadExecutor-4508-thread-1) [    x:collection1] o.a.s.c.PluginBag requestHandler : 'lazy' created with startup=lazy 
   [junit4]   2> 2165835 INFO  (coreLoadExecutor-4508-thread-1) [    x:collection1] o.a.s.c.RequestHandlers Registered paths: /schema,defaults,/get,/config,lazy,mltrh,/replication,tvrh,/debug/dump,/admin/segments,spellCheckCompRH,/update/json,/admin/properties,spellCheckCompRH_Direct,/admin/file,/admin/logging,/terms,/update/json/docs,/search-facet-def,/admin/threads,/search-facet-invariants,/admin/ping,/mlt,standard,/update,/update/csv,/admin/system,mock,spellCheckWithWordbreak,/admin/mbeans,spellCheckWithWordbreak_Direct,/admin/plugins,spellCheckCompRH1,dismax,/admin/luke
   [junit4]   2> 2165837 WARN  (coreLoadExecutor-4508-thread-1) [    x:collection1] o.a.s.c.SolrCore solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
   [junit4]   2> 2165837 INFO  (coreLoadExecutor-4508-thread-1) [    x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 2165838 INFO  (coreLoadExecutor-4508-thread-1) [    x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 2165838 INFO  (coreLoadExecutor-4508-thread-1) [    x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 2165838 INFO  (coreLoadExecutor-4508-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=29, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
   [junit4]   2> 2165839 INFO  (coreLoadExecutor-4508-thread-1) [    x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(SimpleFSDirectory@/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.TestDistributedSearch_DA3F7411FFE8837F-001/index-SimpleFSDirectory-004 lockFactory=org.apache.lucene.store.NativeFSLockFactory@29f68ad9),segFN=segments_1,generation=1}
   [junit4]   2> 2165839 INFO  (coreLoadExecutor-4508-thread-1) [    x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 2165839 INFO  (coreLoadExecutor-4508-thread-1) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@55bb101b[collection1] main
   [junit4]   2> 2165840 INFO  (coreLoadExecutor-4508-thread-1) [    x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {storageIO=org.apache.solr.rest.ManagedResourceStorage$InMemoryStorageIO}
   [junit4]   2> 2165840 INFO  (coreLoadExecutor-4508-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using InMemoryStorage
   [junit4]   2> 2165840 INFO  (coreLoadExecutor-4508-thread-1) [    x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 2165840 INFO  (coreLoadExecutor-4508-thread-1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Initializing spell checkers
   [junit4]   2> 2165848 INFO  (coreLoadExecutor-4508-thread-1) [    x:collection1] o.a.s.s.DirectSolrSpellChecker init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
   [junit4]   2> 2165878 INFO  (coreLoadExecutor-4508-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 2165878 INFO  (searcherExecutor-4509-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: default
   [junit4]   2> 2165879 INFO  (searcherExecutor-4509-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: direct
   [junit4]   2> 2165879 INFO  (searcherExecutor-4509-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: wordbreak
   [junit4]   2> 2165879 INFO  (searcherExecutor-4509-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: multipleFields
   [junit4]   2> 2165879 INFO  (searcherExecutor-4509-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: jarowinkler
   [junit4]   2> 2165880 INFO  (searcherExecutor-4509-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: external
   [junit4]   2> 2165880 INFO  (coreLoadExecutor-4508-thread-1) [    x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 2165880 INFO  (searcherExecutor-4509-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: freq
   [junit4]   2> 2165880 INFO  (searcherExecutor-4509-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: fqcn
   [junit4]   2> 2165881 INFO  (searcherExecutor-4509-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: perDict
   [junit4]   2> 2165882 INFO  (searcherExecutor-4509-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@55bb101b[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2165978 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    ] o.a.s.SolrTestCaseJ4 ###Starting test
   [junit4]   2> 2165981 INFO  (qtp738645470-14091) [    x:collection1] o.a.s.c.SolrCore [collection1] REMOVING ALL DOCUMENTS FROM INDEX
   [junit4]   2> 2165981 INFO  (qtp738645470-14091) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {deleteByQuery=*:*} 0 0
   [junit4]   2> 2165983 INFO  (qtp108976334-14109) [    x:collection1] o.a.s.c.SolrCore [collection1] REMOVING ALL DOCUMENTS FROM INDEX
   [junit4]   2> 2165983 INFO  (qtp108976334-14109) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {deleteByQuery=*:*} 0 0
   [junit4]   2> 2165988 INFO  (qtp738645470-14092) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[1]} 0 2
   [junit4]   2> 2165992 INFO  (qtp108976334-14110) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[1]} 0 2
   [junit4]   2> 2165995 INFO  (qtp738645470-14093) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[2]} 0 1
   [junit4]   2> 2165998 INFO  (qtp108976334-14111) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[2]} 0 1
   [junit4]   2> 2166001 INFO  (qtp738645470-14094) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[3]} 0 0
   [junit4]   2> 2166004 INFO  (qtp108976334-14112) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[3]} 0 0
   [junit4]   2> 2166007 INFO  (qtp738645470-14095) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[4]} 0 1
   [junit4]   2> 2166010 INFO  (qtp108976334-14113) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[4]} 0 0
   [junit4]   2> 2166013 INFO  (qtp738645470-14091) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[5]} 0 0
   [junit4]   2> 2166015 INFO  (qtp108976334-14109) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[5]} 0 0
   [junit4]   2> 2166018 INFO  (qtp738645470-14092) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[6]} 0 0
   [junit4]   2> 2166021 INFO  (qtp108976334-14110) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[6]} 0 0
   [junit4]   2> 2166023 INFO  (qtp738645470-14093) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[7]} 0 0
   [junit4]   2> 2166026 INFO  (qtp108976334-14111) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[7]} 0 0
   [junit4]   2> 2166028 INFO  (qtp738645470-14094) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[8]} 0 0
   [junit4]   2> 2166031 INFO  (qtp108976334-14112) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[8]} 0 0
   [junit4]   2> 2166034 INFO  (qtp738645470-14095) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[9]} 0 0
   [junit4]   2> 2166036 INFO  (qtp108976334-14113) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[9]} 0 0
   [junit4]   2> 2166038 INFO  (qtp738645470-14091) [    x:collection1] o.a.s.u.UpdateHandler start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 2166046 INFO  (qtp738645470-14091) [    x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(SimpleFSDirectory@/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.TestDistributedSearch_DA3F7411FFE8837F-001/index-SimpleFSDirectory-002 lockFactory=org.apache.lucene.store.NativeFSLockFactory@29f68ad9),segFN=segments_1,generation=1}
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(SimpleFSDirectory@/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.TestDistributedSearch_DA3F7411FFE8837F-001/index-SimpleFSDirectory-002 lockFactory=org.apache.lucene.store.NativeFSLockFactory@29f68ad9),segFN=segments_2,generation=2}
   [junit4]   2> 2166047 INFO  (qtp738645470-14091) [    x:collection1] o.a.s.c.SolrCore newest commit generation = 2
   [junit4]   2> 2166050 INFO  (qtp738645470-14091) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@56fc74e2[collection1] main
   [junit4]   2> 2166050 INFO  (qtp738645470-14091) [    x:collection1] o.a.s.u.UpdateHandler end_commit_flush
   [junit4]   2> 2166052 INFO  (searcherExecutor-4499-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@56fc74e2[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(5.5.0):C9)))}
   [junit4]   2> 2166052 INFO  (qtp738645470-14091) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={softCommit=false&commit=true&version=2&wt=javabin&waitSearcher=true} {commit=} 0 13
   [junit4]   2> 2166054 INFO  (qtp108976334-14109) [    x:collection1] o.a.s.u.UpdateHandler start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 2166062 INFO  (qtp108976334-14109) [    x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(SimpleFSDirectory@/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.TestDistributedSearch_DA3F7411FFE8837F-001/index-SimpleFSDirectory-004 lockFactory=org.apache.lucene.store.NativeFSLockFactory@29f68ad9),segFN=segments_1,generation=1}
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(SimpleFSDirectory@/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.TestDistributedSearch_DA3F7411FFE8837F-001/index-SimpleFSDirectory-004 lockFactory=org.apache.lucene.store.NativeFSLockFactory@29f68ad9),segFN=segments_2,generation=2}
   [junit4]   2> 2166062 INFO  (qtp108976334-14109) [    x:collection1] o.a.s.c.SolrCore newest commit generation = 2
   [junit4]   2> 2166065 INFO  (qtp108976334-14109) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@1b16b40d[collection1] main
   [junit4]   2> 2166065 INFO  (qtp108976334-14109) [    x:collection1] o.a.s.u.UpdateHandler end_commit_flush
   [junit4]   2> 2166066 INFO  (searcherExecutor-4509-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@1b16b40d[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(5.5.0):C9)))}
   [junit4]   2> 2166067 INFO  (qtp108976334-14109) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={softCommit=false&commit=true&version=2&wt=javabin&waitSearcher=true} {commit=} 0 12
   [junit4]   2> 2166070 INFO  (qtp738645470-14092) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[10]} 0 1
   [junit4]   2> 2166074 INFO  (qtp108976334-14110) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[10]} 0 1
   [junit4]   2> 2166077 INFO  (qtp738645470-14093) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[11]} 0 0
   [junit4]   2> 2166079 INFO  (qtp108976334-14111) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[11]} 0 0
   [junit4]   2> 2166082 INFO  (qtp738645470-14094) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[12]} 0 0
   [junit4]   2> 2166085 INFO  (qtp108976334-14112) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[12]} 0 0
   [junit4]   2> 2166087 INFO  (qtp738645470-14095) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[13]} 0 0
   [junit4]   2> 2166090 INFO  (qtp108976334-14113) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[13]} 0 0
   [junit4]   2> 2166093 INFO  (qtp738645470-14091) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[1001]} 0 0
   [junit4]   2> 2166096 INFO  (qtp108976334-14109) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[1001]} 0 1
   [junit4]   2> 2166098 INFO  (qtp738645470-14092) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[14]} 0 0
   [junit4]   2> 2166100 INFO  (qtp108976334-14110) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[14]} 0 0
   [junit4]   2> 2166103 INFO  (qtp738645470-14093) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[15]} 0 0
   [junit4]   2> 2166106 INFO  (qtp108976334-14111) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[15]} 0 0
   [junit4]   2> 2166108 INFO  (qtp738645470-14094) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[16]} 0 0
   [junit4]   2> 2166110 INFO  (qtp108976334-14112) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[16]} 0 0
   [junit4]   2> 2166114 INFO  (qtp738645470-14095) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[17]} 0 1
   [junit4]   2> 2166118 INFO  (qtp108976334-14113) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[17]} 0 1
   [junit4]   2> 2166121 INFO  (qtp738645470-14091) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[100]} 0 0
   [junit4]   2> 2166123 INFO  (qtp108976334-14109) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[100]} 0 0
   [junit4]   2> 2166126 INFO  (qtp738645470-14092) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[101]} 0 0
   [junit4]   2> 2166129 INFO  (qtp108976334-14110) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[101]} 0 0
   [junit4]   2> 2166132 INFO  (qtp738645470-14093) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[102]} 0 0
   [junit4]   2> 2166134 INFO  (qtp108976334-14111) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[102]} 0 0
   [junit4]   2> 2166137 INFO  (qtp738645470-14094) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[103]} 0 0
   [junit4]   2> 2166139 INFO  (qtp108976334-14112) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[103]} 0 0
   [junit4]   2> 2166141 INFO  (qtp738645470-14095) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[104]} 0 0
   [junit4]   2> 2166144 INFO  (qtp108976334-14113) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[104]} 0 0
   [junit4]   2> 2166148 INFO  (qtp738645470-14091) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[105]} 0 1
   [junit4]   2> 2166150 INFO  (qtp108976334-14109) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[105]} 0 0
   [junit4]   2> 2166153 INFO  (qtp738645470-14092) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[106]} 0 0
   [junit4]   2> 2166155 INFO  (qtp108976334-14110) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[106]} 0 0
   [junit4]   2> 2166158 INFO  (qtp738645470-14093) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[107]} 0 0
   [junit4]   2> 2166160 INFO  (qtp108976334-14111) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[107]} 0 0
   [junit4]   2> 2166163 INFO  (qtp738645470-14094) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[108]} 0 0
   [junit4]   2> 2166165 INFO  (qtp108976334-14112) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[108]} 0 0
   [junit4]   2> 2166168 INFO  (qtp738645470-14095) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[109]} 0 0
   [junit4]   2> 2166172 INFO  (qtp108976334-14113) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[109]} 0 2
   [junit4]   2> 2166175 INFO  (qtp738645470-14091) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[110]} 0 0
   [junit4]   2> 2166178 INFO  (qtp108976334-14109) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[110]} 0 0
   [junit4]   2> 2166180 INFO  (qtp738645470-14092) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[111]} 0 0
   [junit4]   2> 2166183 INFO  (qtp108976334-14110) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[111]} 0 0
   [junit4]   2> 2166185 INFO  (qtp738645470-14093) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[112]} 0 0
   [junit4]   2> 2166187 INFO  (qtp108976334-14111) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[112]} 0 0
   [junit4]   2> 2166190 INFO  (qtp738645470-14094) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[113]} 0 0
   [junit4]   2> 2166192 INFO  (qtp108976334-14112) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[113]} 0 0
   [junit4]   2> 2166195 INFO  (qtp738645470-14095) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[114]} 0 0
   [junit4]   2> 2166197 INFO  (qtp108976334-14113) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[114]} 0 0
   [junit4]   2> 2166199 INFO  (qtp738645470-14091) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[115]} 0 0
   [junit4]   2> 2166202 INFO  (qtp108976334-14109) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[115]} 0 0
   [junit4]   2> 2166204 INFO  (qtp738645470-14092) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[116]} 0 0
   [junit4]   2> 2166207 INFO  (qtp108976334-14110) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[116]} 0 0
   [junit4]   2> 2166209 INFO  (qtp738645470-14093) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[117]} 0 0
   [junit4]   2> 2166211 INFO  (qtp108976334-14111) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[117]} 0 0
   [junit4]   2> 2166214 INFO  (qtp738645470-14094) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[118]} 0 0
   [junit4]   2> 2166216 INFO  (qtp108976334-14112) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[118]} 0 0
   [junit4]   2> 2166219 INFO  (qtp738645470-14095) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[119]} 0 0
   [junit4]   2> 2166221 INFO  (qtp108976334-14113) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[119]} 0 0
   [junit4]   2> 2166223 INFO  (qtp738645470-14091) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[120]} 0 0
   [junit4]   2> 2166226 INFO  (qtp108976334-14109) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[120]} 0 0
   [junit4]   2> 2166228 INFO  (qtp738645470-14092) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[121]} 0 0
   [junit4]   2> 2166231 INFO  (qtp108976334-14110) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[121]} 0 0
   [junit4]   2> 2166233 INFO  (qtp738645470-14093) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[122]} 0 0
   [junit4]   2> 2166235 INFO  (qtp108976334-14111) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[122]} 0 0
   [junit4]   2> 2166238 INFO  (qtp738645470-14094) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[123]} 0 0
   [junit4]   2> 2166240 INFO  (qtp108976334-14112) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[123]} 0 0
   [junit4]   2> 2166243 INFO  (qtp738645470-14095) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[124]} 0 0
   [junit4]   2> 2166245 INFO  (qtp108976334-14113) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[124]} 0 0
   [junit4]   2> 2166248 INFO  (qtp738645470-14091) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[125]} 0 0
   [junit4]   2> 2166250 INFO  (qtp108976334-14109) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[125]} 0 0
   [junit4]   2> 2166253 INFO  (qtp738645470-14092) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[126]} 0 0
   [junit4]   2> 2166255 INFO  (qtp108976334-14110) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[126]} 0 0
   [junit4]   2> 2166258 INFO  (qtp738645470-14093) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[127]} 0 0
   [junit4]   2> 2166260 INFO  (qtp108976334-14111) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[127]} 0 0
   [junit4]   2> 2166262 INFO  (qtp738645470-14094) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[128]} 0 0
   [junit4]   2> 2166265 INFO  (qtp108976334-14112) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[128]} 0 0
   [junit4]   2> 2166267 INFO  (qtp738645470-14095) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[129]} 0 0
   [junit4]   2> 2166269 INFO  (qtp108976334-14113) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[129]} 0 0
   [junit4]   2> 2166272 INFO  (qtp738645470-14091) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[130]} 0 0
   [junit4]   2> 2166274 INFO  (qtp108976334-14109) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[130]} 0 0
   [junit4]   2> 2166277 INFO  (qtp738645470-14092) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[131]} 0 0
   [junit4]   2> 2166279 INFO  (qtp108976334-14110) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[131]} 0 0
   [junit4]   2> 2166281 INFO  (qtp738645470-14093) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[132]} 0 0
   [junit4]   2> 2166284 INFO  (qtp108976334-14111) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[132]} 0 0
   [junit4]   2> 2166286 INFO  (qtp738645470-14094) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[133]} 0 0
   [junit4]   2> 2166289 INFO  (qtp108976334-14112) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[133]} 0 0
   [junit4]   2> 2166291 INFO  (qtp738645470-14095) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[134]} 0 0
   [junit4]   2> 2166293 INFO  (qtp108976334-14113) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[134]} 0 0
   [junit4]   2> 2166296 INFO  (qtp738645470-14091) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[135]} 0 0
   [junit4]   2> 2166298 INFO  (qtp108976334-14109) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[135]} 0 0
   [junit4]   2> 2166300 INFO  (qtp738645470-14092) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[136]} 0 0
   [junit4]   2> 2166303 INFO  (qtp108976334-14110) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[136]} 0 0
   [junit4]   2> 2166305 INFO  (qtp738645470-14093) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[137]} 0 0
   [junit4]   2> 2166308 INFO  (qtp108976334-14111) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[137]} 0 0
   [junit4]   2> 2166310 INFO  (qtp738645470-14094) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[138]} 0 0
   [junit4]   2> 2166313 INFO  (qtp108976334-14112) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[138]} 0 0
   [junit4]   2> 2166317 INFO  (qtp738645470-14095) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[139]} 0 0
   [junit4]   2> 2166320 INFO  (qtp108976334-14113) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[139]} 0 0
   [junit4]   2> 2166322 INFO  (qtp738645470-14091) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[140]} 0 0
   [junit4]   2> 2166325 INFO  (qtp108976334-14109) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[140]} 0 0
   [junit4]   2> 2166327 INFO  (qtp738645470-14092) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[141]} 0 0
   [junit4]   2> 2166330 INFO  (qtp108976334-14110) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[141]} 0 0
   [junit4]   2> 2166333 INFO  (qtp738645470-14093) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[142]} 0 0
   [junit4]   2> 2166335 INFO  (qtp108976334-14111) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[142]} 0 0
   [junit4]   2> 2166338 INFO  (qtp738645470-14094) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[143]} 0 0
   [junit4]   2> 2166340 INFO  (qtp108976334-14112) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[143]} 0 0
   [junit4]   2> 2166342 INFO  (qtp738645470-14095) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[144]} 0 0
   [junit4]   2> 2166345 INFO  (qtp108976334-14113) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[144]} 0 0
   [junit4]   2> 2166347 INFO  (qtp738645470-14091) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[145]} 0 0
   [junit4]   2> 2166350 INFO  (qtp108976334-14109) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[145]} 0 0
   [junit4]   2> 2166352 INFO  (qtp738645470-14092) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[146]} 0 0
   [junit4]   2> 2166355 INFO  (qtp108976334-14110) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[146]} 0 0
   [junit4]   2> 2166358 INFO  (qtp738645470-14093) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[147]} 0 0
   [junit4]   2> 2166361 INFO  (qtp108976334-14111) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[147]} 0 0
   [junit4]   2> 2166364 INFO  (qtp738645470-14094) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[148]} 0 0
   [junit4]   2> 2166367 INFO  (qtp108976334-14112) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[148]} 0 0
   [junit4]   2> 2166369 INFO  (qtp738645470-14095) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[149]} 0 0
   [junit4]   2> 2166372 INFO  (qtp108976334-14113) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[149]} 0 0
   [junit4]   2> 2166374 INFO  (qtp738645470-14091) [    x:collection1] o.a.s.u.UpdateHandler start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 2166393 INFO  (qtp738645470-14091) [    x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(SimpleFSDirectory@/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.TestDistributedSearch_DA3F7411FFE8837F-001/index-SimpleFSDirectory-002 lockFactory=org.apache.lucene.store.NativeFSLockFactory@29f68ad9),segFN=segments_2,generation=2}
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(SimpleFSDirectory@/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.TestDistributedSearch_DA3F7411FFE8837F-001/index-SimpleFSDirectory-002 lockFactory=org.apache.lucene.store.NativeFSLockFactory@29f68ad9),segFN=segments_3,generation=3}
   [junit4]   2> 2166393 INFO  (qtp738645470-14091) [    x:collection1] o.a.s.c.SolrCore newest commit generation = 3
   [junit4]   2> 2166394 INFO  (qtp738645470-14091) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@7c1a7d46[collection1] main
   [junit4]   2> 2166394 INFO  (qtp738645470-14091) [    x:collection1] o.a.s.u.UpdateHandler end_commit_flush
   [junit4]   2> 2166396 INFO  (searcherExecutor-4499-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@7c1a7d46[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(5.5.0):C9) Uninverting(_1(5.5.0):C59)))}
   [junit4]   2> 2166396 INFO  (qtp738645470-14091) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={softCommit=false&commit=true&version=2&wt=javabin&waitSearcher=true} {commit=} 0 21
   [junit4]   2> 2166398 INFO  (qtp108976334-14109) [    x:collection1] o.a.s.u.UpdateHandler start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 2166414 INFO  (qtp108976334-14109) [    x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(SimpleFSDirectory@/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.TestDistributedSearch_DA3F7411FFE8837F-001/index-SimpleFSDirectory-004 lockFactory=org.apache.lucene.store.NativeFSLockFactory@29f68ad9),segFN=segments_2,generation=2}
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(SimpleFSDirectory@/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.TestDistributedSearch_DA3F7411FFE8837F-001/index-SimpleFSDirectory-004 lockFactory=org.apache.lucene.store.NativeFSLockFactory@29f68ad9),segFN=segments_3,generation=3}
   [junit4]   2> 2166414 INFO  (qtp108976334-14109) [    x:collection1] o.a.s.c.SolrCore newest commit generation = 3
   [junit4]   2> 2166415 INFO  (qtp108976334-14109) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@77abb39e[collection1] main
   [junit4]   2> 2166415 INFO  (qtp108976334-14109) [    x:collection1] o.a.s.u.UpdateHandler end_commit_flush
   [junit4]   2> 2166416 INFO  (searcherExecutor-4509-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@77abb39e[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(5.5.0):C9) Uninverting(_1(5.5.0):C59)))}
   [junit4]   2> 2166417 INFO  (qtp108976334-14109) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp= path=/update params={softCommit=false&commit=true&version=2&wt=javabin&waitSearcher=true} {commit=} 0 18
   [junit4]   2> 2166419 INFO  (qtp738645470-14092) [    x:collection1] o.a.s.c.SolrCore Ignoring exception matching parameter cannot be negative
   [junit4]   2> 2166419 INFO  (qtp738645470-14092) [    x:collection1] o.a.s.c.S.Request [collection1] webapp= path=/select params={start=-1&q=*&version=2&wt=javabin&distrib=false} status=400 QTime=0 
   [junit4]   2> 2166422 INFO  (qtp738645470-14093) [    x:collection1] o.a.s.c.SolrCore Ignoring exception matching parameter cannot be negative
   [junit4]   2> 2166422 INFO  (qtp738645470-14093) [    x:collection1] o.a.s.c.S.Request [collection1] webapp= path=/select params={start=0&q=*&version=2&wt=javabin&distrib=false&rows=-1} status=400 QTime=0 
   [junit4]   2> 2166425 INFO  (qtp738645470-14094) [    x:collection1] o.a.s.c.S.Request [collection1] webapp= path=/select params={sort=n_ti1+desc&q=*:*&version=2&wt=javabin&distrib=false} hits=68 status=0 QTime=1 
   [junit4]   2> 2166509 INFO  (qtp108976334-14111) [    x:collection1] o.a.s.c.S.Request [collection1] webapp= path=/select params={isShard=true&q=*:*&version=2&rows=10&distrib=false&shard.url=[ff01::083]:33332/|http://127.0.0.1:55549//collection1|[ff01::114]:33332/|[ff01::213]:33332/&sort=n_ti1+desc&NOW=1448720185112&fl=id&start=0&shards.purpose=4&wt=javabin&fsv=true} hits=68 status=0 QTime=1 
   [junit4]   2> 2166514 INFO  (qtp108976334-14112) [    x:collection1] o.a.s.c.S.Request [collection1] webapp= path=/select params={NOW=1448720185112&ids=132,135,7,2,119,102,142,145,147,105&isShard=true&shards.purpose=64&q=*:*&version=2&wt=javabin&shard.url=[ff01::083]:33332/|http://127.0.0.1:55549//collection1|[ff01::114]:33332/|[ff01::213]:33332/&distrib=false} status=0 QTime=0 
   [junit4]   2> 2166518 INFO  (qtp108976334-14110) [    x:collection1] o.a.s.c.S.Request [collection1] webapp= path=/select params={sort=n_ti1+desc&q=*:*&version=2&wt=javabin&shards=[ff01::083]:33332/|http://127.0.0.1:55549//collection1|[ff01::114]:33332/|[ff01::213]:33332/} hits=68 status=0 QTime=88 
   [junit4]   2> 2166519 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    ] o.a.s.BaseDistributedSearchTestCase starting stress...
   [junit4]   2> 2166523 INFO  (qtp108976334-14112) [    x:collection1] o.a.s.c.S.Request [collection1] webapp= path=/select params={isShard=true&q=*:*&version=2&rows=10&distrib=false&shard.url=[ff01::083]:33332/|http://127.0.0.1:55549//collection1|[ff01::114]:33332/|[ff01::213]:33332/&sort=n_ti1+desc&NOW=1448720185203&fl=id&start=0&shards.purpose=4&wt=javabin&fsv=true} hits=68 status=0 QTime=0 
   [junit4]   2> 2166526 INFO  (qtp108976334-14110) [    x:collection1] o.a.s.c.S.Request [collection1] webapp= path=/select params={NOW=1448720185203&ids=132,135,7,2,119,102,142,145,147,105&isShard=true&shards.purpose=64&q=*:*&version=2&wt=javabin&shard.url=[ff01::083]:33332/|http://127.0.0.1:55549//collection1|[ff01::114]:33332/|[ff01::213]:33332/&distrib=false} status=0 QTime=0 
   [junit4]   2> 2166527 INFO  (qtp108976334-14113) [    x:collection1] o.a.s.c.S.Request [collection1] webapp= path=/select params={sort=n_ti1+desc&q=*:*&version=2&wt=javabin&shards=[ff01::083]:33332/|http://127.0.0.1:55549//collection1|[ff01::114]:33332/|[ff01::213]:33332/} hits=68 status=0 QTime=6 
   [junit4]   2> 2166704 INFO  (qtp108976334-14110) [    x:collection1] o.a.s.c.S.Request [collection1] webapp= path=/select params={isShard=true&q=*:*&version=2&rows=10&distrib=false&shard.url=[ff01::083]:33332/|http://127.0.0.1:55549//collection1|[ff01::114]:33332/|[ff01::213]:33332/&sort=n_ti1+desc&NOW=1448720185383&fl=id&start=0&shards.purpose=4&wt=javabin&fsv=true} hits=68 status=0 QTime=0 
   [junit4]   2> 2166707 INFO  (qtp108976334-14113) [    x:collection1] o.a.s.c.S.Request [collection1] webapp= path=/select params={NOW=1448720185383&ids=132,135,7,2,119,102,142,145,147,105&isShard=true&shards.purpose=64&q=*:*&version=2&wt=javabin&shard.url=[ff01::083]:33332/|http://127.0.0.1:55549//collection1|[ff01::114]:33332/|[ff01::213]:33332/&distrib=false} status=0 QTime=0 
   [junit4]   2> 2166707 INFO  (qtp108976334-14112) [    x:collection1] o.a.s.c.S.Request [collection1] webapp= path=/select params={sort=n_ti1+desc&q=*:*&version=2&wt=javabin&shards=[ff01::083]:33332/|http://127.0.0.1:55549//collection1|[ff01::114]:33332/|[ff01::213]:33332/} hits=68 status=0 QTime=6 
   [junit4]   2> 2166776 INFO  (qtp108976334-14110) [    x:collection1] o.a.s.c.S.Request [collection1] webapp= path=/select params={isShard=true&q=*:*&version=2&rows=10&distrib=false&shard.url=[ff01::083]:33332/|http://127.0.0.1:55549//collection1|[ff01::114]:33332/|[ff01::213]:33332/&sort=n_ti1+desc&NOW=1448720185203&fl=id&start=0&shards.purpose=4&wt=javabin&fsv=true} hits=68 status=0 QTime=0 
   [junit4]   2> 2166780 INFO  (qtp108976334-14113) [    x:collection1] o.a.s.c.S.Request [collection1] webapp= path=/select params={NOW=1448720185203&ids=132,135,7,2,119,102,142,145,147,105&isShard=true&shards.purpose=64&q=*:*&version=2&wt=javab

[...truncated too long message...]

s/collection1/data/index;done=false>>]
   [junit4]   2> 2268491 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.TestDistributedSearch_DA3F7411FFE8837F-001/tempDir-001/shard0/cores/collection1/data/index
   [junit4]   2> 2268493 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    x:collection1] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@2bf26ef8{/,null,UNAVAILABLE}
   [junit4]   2> 2268500 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    ] o.e.j.s.ServerConnector Stopped ServerConnector@1617e798{HTTP/1.1}{127.0.0.1:0}
   [junit4]   2> 2268500 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1743729531
   [junit4]   2> 2268500 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    x:collection1] o.a.s.c.SolrCore [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@4681538b
   [junit4]   2> 2268501 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    x:collection1] o.a.s.u.UpdateHandler closing DirectUpdateHandler2{commits=2,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=34,cumulative_deletesById=0,cumulative_deletesByQuery=1,cumulative_errors=0}
   [junit4]   2> 2268501 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    x:collection1] o.a.s.u.SolrCoreState Closing SolrCoreState
   [junit4]   2> 2268501 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    x:collection1] o.a.s.u.DefaultSolrCoreState SolrCoreState ref count has reached 0 - closing IndexWriter
   [junit4]   2> 2268501 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    x:collection1] o.a.s.u.DefaultSolrCoreState closing IndexWriter with IndexWriterCloser
   [junit4]   2> 2268502 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    x:collection1] o.a.s.c.SolrCore [collection1] Closing main searcher on request.
   [junit4]   2> 2268516 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing MockDirectoryFactory - 2 directories currently being tracked
   [junit4]   2> 2268516 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    x:collection1] o.a.s.c.CachingDirectoryFactory looking to close /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.TestDistributedSearch_DA3F7411FFE8837F-001/tempDir-001/shard1/cores/collection1/data/index [CachedDir<<refCount=0;path=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.TestDistributedSearch_DA3F7411FFE8837F-001/tempDir-001/shard1/cores/collection1/data/index;done=false>>]
   [junit4]   2> 2268516 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.TestDistributedSearch_DA3F7411FFE8837F-001/tempDir-001/shard1/cores/collection1/data/index
   [junit4]   2> 2268516 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    x:collection1] o.a.s.c.CachingDirectoryFactory looking to close /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.TestDistributedSearch_DA3F7411FFE8837F-001/tempDir-001/shard1/cores/collection1/data [CachedDir<<refCount=0;path=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.TestDistributedSearch_DA3F7411FFE8837F-001/tempDir-001/shard1/cores/collection1/data;done=false>>]
   [junit4]   2> 2268516 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.TestDistributedSearch_DA3F7411FFE8837F-001/tempDir-001/shard1/cores/collection1/data
   [junit4]   2> 2268517 INFO  (TEST-TestDistributedSearch.test-seed#[DA3F7411FFE8837F]) [    x:collection1] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@5307e9ac{/,null,UNAVAILABLE}
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestDistributedSearch -Dtests.method=test -Dtests.seed=DA3F7411FFE8837F -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=es_MX -Dtests.timezone=America/Maceio -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR    104s J2 | TestDistributedSearch.test <<<
   [junit4]    > Throwable #1: java.lang.NullPointerException
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([DA3F7411FFE8837F:526B4BCB5114EE87]:0)
   [junit4]    > 	at org.apache.solr.TestDistributedSearch.test(TestDistributedSearch.java:568)
   [junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsRepeatStatement.callStatement(BaseDistributedSearchTestCase.java:986)
   [junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:939)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 2268521 INFO  (SUITE-TestDistributedSearch-seed#[DA3F7411FFE8837F]-worker) [    ] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.TestDistributedSearch_DA3F7411FFE8837F-001
   [junit4]   2> NOTE: test params are: codec=CheapBastard, sim=RandomSimilarityProvider(queryNorm=false,coord=crazy): {}, locale=es_MX, timezone=America/Maceio
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_80 (64-bit)/cpus=4,threads=1,free=205244096,total=487587840
   [junit4]   2> NOTE: All tests run in this JVM: [QueryParsingTest, TestInfoStreamLogging, TestLeaderElectionZkExpiry, TestJsonRequest, TestReloadDeadlock, TestOrdValues, TestManagedSchemaDynamicFieldResource, TestSolrDeletionPolicy1, ScriptEngineTest, ShowFileRequestHandlerTest, TestReversedWildcardFilterFactory, TestZkChroot, TestSuggestSpellingConverter, UpdateParamsTest, TestRandomRequestDistribution, TestIndexSearcher, TestJmxMonitoredMap, TestMiniSolrCloudCluster, TestGroupingSearch, FieldAnalysisRequestHandlerTest, MultiThreadedOCPTest, TestComplexPhraseQParserPlugin, SolrXmlInZkTest, LeaderElectionTest, TestSolrJ, TestIntervalFaceting, DistributedFacetPivotSmallTest, DirectSolrConnectionTest, TestSchemaVersionResource, BlockCacheTest, MergeStrategyTest, ForceLeaderTest, DistributedQueryComponentCustomSortTest, HardAutoCommitTest, TestRestoreCore, SliceStateTest, AssignTest, DocExpirationUpdateProcessorFactoryTest, SparseHLLTest, RegexBoostProcessorTest, PluginInfoTest, CollectionStateFormat2Test, TestSolrIndexConfig, TestRandomCollapseQParserPlugin, PrimitiveFieldTypeTest, SuggesterTest, TestStressRecovery, TestFileDictionaryLookup, RemoteQueryErrorTest, DirectUpdateHandlerOptimizeTest, RequestHandlersTest, BlockDirectoryTest, QueryResultKeyTest, SuggesterTSTTest, BasicDistributedZk2Test, CoreAdminRequestStatusTest, TestConfigOverlay, TestConfigSetsAPI, TestTrackingShardHandlerFactory, OverseerTest, CollectionsAPIAsyncDistributedZkTest, OverseerStatusTest, TestScoreJoinQPScore, CurrencyFieldXmlFileTest, SharedFSAutoReplicaFailoverUtilsTest, AliasIntegrationTest, IgnoreCommitOptimizeUpdateProcessorFactoryTest, TestAuthenticationFramework, UnloadDistributedZkTest, CustomCollectionTest, HighlighterConfigTest, TestIndexingPerformance, TestReplicaProperties, TestMiniSolrCloudClusterBase, TestHashQParserPlugin, HdfsSyncSliceTest, FieldMutatingUpdateProcessorTest, TestCloudManagedSchemaConcurrent, TestBlendedInfixSuggestions, AddSchemaFieldsUpdateProcessorFactoryTest, DirectUpdateHandlerTest, TestCloudInspectUtil, TestCharFilters, AtomicUpdatesTest, TestCoreDiscovery, DateRangeFieldTest, TestNoOpRegenerator, TestExactSharedStatsCache, TestExtendedDismaxParser, ClusterStateUpdateTest, TestRandomDVFaceting, TestDistributedSearch]
   [junit4] Completed [295/551] on J2 in 104.16s, 1 test, 1 error <<< FAILURES!

[...truncated 832 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 551 suites (7 ignored), 2177 tests, 1 error, 32 ignored (18 assumptions) [seed: DA3F7411FFE8837F]

Total time: 183 minutes 39 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1029 - Still Failing

[Apache Jenkins Server <je...@builds.apache.org>]

Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1029/

1 tests failed.
FAILED:  org.apache.solr.cloud.hdfs.HdfsBasicDistributedZkTest.test

Error Message:
commitWithin did not work on node: http://127.0.0.1:41067/jv_rk/collection1 expected:<68> but was:<67>

Stack Trace:
java.lang.AssertionError: commitWithin did not work on node: http://127.0.0.1:41067/jv_rk/collection1 expected:<68> but was:<67>
	at __randomizedtesting.SeedInfo.seed([AF61550CE06C52F5:27356AD64E903F0D]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.failNotEquals(Assert.java:647)
	at org.junit.Assert.assertEquals(Assert.java:128)
	at org.junit.Assert.assertEquals(Assert.java:472)
	at org.apache.solr.cloud.BasicDistributedZkTest.test(BasicDistributedZkTest.java:333)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:964)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:939)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 11454 lines...]
   [junit4] Suite: org.apache.solr.cloud.hdfs.HdfsBasicDistributedZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsBasicDistributedZkTest_AF61550CE06C52F5-001/init-core-data-001
   [junit4]   2> 3765093 INFO  (SUITE-HdfsBasicDistributedZkTest-seed#[AF61550CE06C52F5]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /jv_rk/
   [junit4]   1> Formatting using clusterid: testClusterID
   [junit4]   2> 3765166 WARN  (SUITE-HdfsBasicDistributedZkTest-seed#[AF61550CE06C52F5]-worker) [    ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
   [junit4]   2> 3765179 WARN  (SUITE-HdfsBasicDistributedZkTest-seed#[AF61550CE06C52F5]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 3765185 INFO  (SUITE-HdfsBasicDistributedZkTest-seed#[AF61550CE06C52F5]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 3765213 INFO  (SUITE-HdfsBasicDistributedZkTest-seed#[AF61550CE06C52F5]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/hdfs to ./temp/Jetty_localhost_58696_hdfs____p0ucca/webapp
   [junit4]   2> 3765429 INFO  (SUITE-HdfsBasicDistributedZkTest-seed#[AF61550CE06C52F5]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 3765802 INFO  (SUITE-HdfsBasicDistributedZkTest-seed#[AF61550CE06C52F5]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:58696
   [junit4]   2> 3766024 WARN  (SUITE-HdfsBasicDistributedZkTest-seed#[AF61550CE06C52F5]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 3766034 INFO  (SUITE-HdfsBasicDistributedZkTest-seed#[AF61550CE06C52F5]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 3766064 INFO  (SUITE-HdfsBasicDistributedZkTest-seed#[AF61550CE06C52F5]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_49743_datanode____.7cuqmg/webapp
   [junit4]   2> 3766361 INFO  (SUITE-HdfsBasicDistributedZkTest-seed#[AF61550CE06C52F5]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 3766770 INFO  (SUITE-HdfsBasicDistributedZkTest-seed#[AF61550CE06C52F5]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:49743
   [junit4]   2> 3766951 WARN  (SUITE-HdfsBasicDistributedZkTest-seed#[AF61550CE06C52F5]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 3766953 INFO  (SUITE-HdfsBasicDistributedZkTest-seed#[AF61550CE06C52F5]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 3766977 INFO  (SUITE-HdfsBasicDistributedZkTest-seed#[AF61550CE06C52F5]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_40513_datanode____.r6gmm6/webapp
   [junit4]   2> 3767038 INFO  (IPC Server handler 2 on 45072) [    ] BlockStateChange BLOCK* processReport: from storage DS-205df9cd-d3d4-43c6-a6f3-2feea85c39cc node DatanodeRegistration(127.0.0.1, datanodeUuid=04deb699-e0b8-4712-9514-a2f7c3f391ac, infoPort=49743, ipcPort=45445, storageInfo=lv=-56;cid=testClusterID;nsid=395734528;c=0), blocks: 0, hasStaleStorages: true, processing time: 0 msecs
   [junit4]   2> 3767038 INFO  (IPC Server handler 2 on 45072) [    ] BlockStateChange BLOCK* processReport: from storage DS-ac0a84a6-bb67-44fb-a723-4ccf84b541fc node DatanodeRegistration(127.0.0.1, datanodeUuid=04deb699-e0b8-4712-9514-a2f7c3f391ac, infoPort=49743, ipcPort=45445, storageInfo=lv=-56;cid=testClusterID;nsid=395734528;c=0), blocks: 0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 3767242 INFO  (SUITE-HdfsBasicDistributedZkTest-seed#[AF61550CE06C52F5]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 3767440 INFO  (SUITE-HdfsBasicDistributedZkTest-seed#[AF61550CE06C52F5]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:40513
   [junit4]   2> 3767953 INFO  (IPC Server handler 9 on 45072) [    ] BlockStateChange BLOCK* processReport: from storage DS-0577c8ab-aa7d-4a02-8452-0f9cecba68b4 node DatanodeRegistration(127.0.0.1, datanodeUuid=71f520ad-6972-46cb-b10e-08747223bbc0, infoPort=40513, ipcPort=42525, storageInfo=lv=-56;cid=testClusterID;nsid=395734528;c=0), blocks: 0, hasStaleStorages: true, processing time: 0 msecs
   [junit4]   2> 3767953 INFO  (IPC Server handler 9 on 45072) [    ] BlockStateChange BLOCK* processReport: from storage DS-19b96dcd-eb77-4930-b2a3-8e208cb3efbc node DatanodeRegistration(127.0.0.1, datanodeUuid=71f520ad-6972-46cb-b10e-08747223bbc0, infoPort=40513, ipcPort=42525, storageInfo=lv=-56;cid=testClusterID;nsid=395734528;c=0), blocks: 0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 3768186 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 3768186 INFO  (Thread-57490) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 3768187 INFO  (Thread-57490) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 3768286 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.ZkTestServer start zk server on port:49674
   [junit4]   2> 3768286 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3768302 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3768321 INFO  (zkCallback-1626-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2bc336cf name:ZooKeeperConnection Watcher:127.0.0.1:49674 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3768321 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3768321 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3768322 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 3768324 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3768336 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3768339 INFO  (zkCallback-1627-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@3bdc04a5 name:ZooKeeperConnection Watcher:127.0.0.1:49674/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3768340 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3768340 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3768340 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 3768341 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 3768343 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 3768345 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 3768348 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 3768348 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 3768350 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 3768350 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 3768352 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 3768352 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 3768353 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 3768353 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 3768355 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 3768355 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 3768356 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 3768357 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 3768362 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 3768362 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 3768363 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 3768363 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 3768364 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 3768365 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 3768366 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 3768366 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 3768367 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 3768367 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 3768811 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsBasicDistributedZkTest_AF61550CE06C52F5-001/control-001/cores/collection1
   [junit4]   2> 3768813 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 3768864 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5e48bd6f{/jv_rk,null,AVAILABLE}
   [junit4]   2> 3768864 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.e.j.s.ServerConnector Started ServerConnector@798cf2c4{HTTP/1.1}{127.0.0.1:50813}
   [junit4]   2> 3768864 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.e.j.s.Server Started @3771672ms
   [junit4]   2> 3768865 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=hdfs://localhost:45072/hdfs__localhost_45072__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-5.x_solr_build_solr-core_test_J0_temp_solr.cloud.hdfs.HdfsBasicDistributedZkTest_AF61550CE06C52F5-001_tempDir-002_control_data, hostContext=/jv_rk, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsBasicDistributedZkTest_AF61550CE06C52F5-001/control-001/cores, hostPort=50813}
   [junit4]   2> 3768865 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@3827de24
   [junit4]   2> 3768865 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsBasicDistributedZkTest_AF61550CE06C52F5-001/control-001'
   [junit4]   2> 3768865 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 3768865 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 3768865 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3768867 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3768873 INFO  (zkCallback-1628-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@37b39f28 name:ZooKeeperConnection Watcher:127.0.0.1:49674/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3768873 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3768874 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3768875 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 3768875 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsBasicDistributedZkTest_AF61550CE06C52F5-001/control-001/solr.xml
   [junit4]   2> 3768887 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsBasicDistributedZkTest_AF61550CE06C52F5-001/control-001/cores
   [junit4]   2> 3768887 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.CoreContainer New CoreContainer 473949852
   [junit4]   2> 3768887 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsBasicDistributedZkTest_AF61550CE06C52F5-001/control-001]
   [junit4]   2> 3768888 WARN  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsBasicDistributedZkTest_AF61550CE06C52F5-001/control-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsBasicDistributedZkTest_AF61550CE06C52F5-001/control-001/lib
   [junit4]   2> 3768888 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 3768889 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 3768889 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 3768890 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 3768890 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:49674/solr
   [junit4]   2> 3768890 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 3768890 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3768891 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3768909 INFO  (zkCallback-1630-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1e273684 name:ZooKeeperConnection Watcher:127.0.0.1:49674 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3768910 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3768910 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3768937 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [n:127.0.0.1:50813_jv_rk    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3768957 INFO  (zkCallback-1631-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@3afbeb1e name:ZooKeeperConnection Watcher:127.0.0.1:49674/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3768957 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [n:127.0.0.1:50813_jv_rk    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3768958 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [n:127.0.0.1:50813_jv_rk    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 3768960 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [n:127.0.0.1:50813_jv_rk    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 3768962 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [n:127.0.0.1:50813_jv_rk    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 3768963 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [n:127.0.0.1:50813_jv_rk    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 3768964 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [n:127.0.0.1:50813_jv_rk    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 3768966 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [n:127.0.0.1:50813_jv_rk    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 3768967 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [n:127.0.0.1:50813_jv_rk    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 3768967 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [n:127.0.0.1:50813_jv_rk    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 3768968 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [n:127.0.0.1:50813_jv_rk    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 3768969 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [n:127.0.0.1:50813_jv_rk    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50813_jv_rk
   [junit4]   2> 3768969 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [n:127.0.0.1:50813_jv_rk    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:50813_jv_rk
   [junit4]   2> 3768970 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [n:127.0.0.1:50813_jv_rk    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 3768971 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [n:127.0.0.1:50813_jv_rk    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 3768972 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [n:127.0.0.1:50813_jv_rk    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 3768972 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [n:127.0.0.1:50813_jv_rk    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94936039073054724-127.0.0.1:50813_jv_rk-n_0000000000
   [junit4]   2> 3768972 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [n:127.0.0.1:50813_jv_rk    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:50813_jv_rk
   [junit4]   2> 3768973 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [n:127.0.0.1:50813_jv_rk    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 3768973 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [n:127.0.0.1:50813_jv_rk    ] o.a.s.c.Overseer Overseer (id=94936039073054724-127.0.0.1:50813_jv_rk-n_0000000000) starting
   [junit4]   2> 3768975 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [n:127.0.0.1:50813_jv_rk    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 3768979 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [n:127.0.0.1:50813_jv_rk    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 3768985 INFO  (OverseerStateUpdate-94936039073054724-127.0.0.1:50813_jv_rk-n_0000000000) [n:127.0.0.1:50813_jv_rk    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 3768986 INFO  (OverseerStateUpdate-94936039073054724-127.0.0.1:50813_jv_rk-n_0000000000) [n:127.0.0.1:50813_jv_rk    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 3768996 INFO  (OverseerCollectionConfigSetProcessor-94936039073054724-127.0.0.1:50813_jv_rk-n_0000000000) [n:127.0.0.1:50813_jv_rk    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 3769005 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [n:127.0.0.1:50813_jv_rk    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 3769034 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [n:127.0.0.1:50813_jv_rk    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 3769035 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [n:127.0.0.1:50813_jv_rk    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 3769035 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [n:127.0.0.1:50813_jv_rk    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsBasicDistributedZkTest_AF61550CE06C52F5-001/control-001/cores
   [junit4]   2> 3769036 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [n:127.0.0.1:50813_jv_rk    ] o.a.s.c.SolrCore Created CoreDescriptor: {collection=control_collection, dataDir=data/, transient=false, loadOnStartup=true, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsBasicDistributedZkTest_AF61550CE06C52F5-001/control-001/cores/collection1/, shard=, config=solrconfig.xml, coreNodeName=, name=collection1, configSetProperties=configsetprops.json, schema=schema.xml, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsBasicDistributedZkTest_AF61550CE06C52F5-001/control-001/cores/collection1}
   [junit4]   2> 3769037 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [n:127.0.0.1:50813_jv_rk    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsBasicDistributedZkTest_AF61550CE06C52F5-001/control-001/cores/collection1/
   [junit4]   2> 3769037 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [n:127.0.0.1:50813_jv_rk    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 3769048 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 3769048 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 3769065 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [n:127.0.0.1:50813_jv_rk    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0
   [junit4]   2> 3769065 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [n:127.0.0.1:50813_jv_rk    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 3769066 INFO  (OverseerStateUpdate-94936039073054724-127.0.0.1:50813_jv_rk-n_0000000000) [n:127.0.0.1:50813_jv_rk    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "state":"down",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "node_name":"127.0.0.1:50813_jv_rk",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:50813/jv_rk",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "collection":"control_collection"} current state version: 0
   [junit4]   2> 3769066 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 3769066 INFO  (OverseerStateUpdate-94936039073054724-127.0.0.1:50813_jv_rk-n_0000000000) [n:127.0.0.1:50813_jv_rk    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "state":"down",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "node_name":"127.0.0.1:50813_jv_rk",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:50813/jv_rk",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "collection":"control_collection"}
   [junit4]   2> 3769067 INFO  (OverseerStateUpdate-94936039073054724-127.0.0.1:50813_jv_rk-n_0000000000) [n:127.0.0.1:50813_jv_rk    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 3769067 INFO  (OverseerStateUpdate-94936039073054724-127.0.0.1:50813_jv_rk-n_0000000000) [n:127.0.0.1:50813_jv_rk    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 3769068 INFO  (zkCallback-1631-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3769085 INFO  (zkCallback-1631-thread-2-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3770067 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 3770067 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 3770067 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 3770067 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 3770068 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 3770068 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsBasicDistributedZkTest_AF61550CE06C52F5-001/control-001/cores/collection1'
   [junit4]   2> 3770068 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 3770068 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 3770070 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 3770076 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 3770104 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.5.0
   [junit4]   2> 3770179 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 3770186 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 3770639 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 3770647 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 3770648 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 3770662 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 3770665 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 3770667 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 3770668 WARN  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 3770668 WARN  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 3770668 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 3770669 WARN  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 3770669 WARN  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 3770670 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0
   [junit4]   2> 3770670 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 3770670 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.HdfsDirectoryFactory
   [junit4]   2> 3770670 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:45072/solr_hdfs_home
   [junit4]   2> 3770670 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
   [junit4]   2> 3770670 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsBasicDistributedZkTest_AF61550CE06C52F5-001/control-001/cores/collection1], dataDir=[null]
   [junit4]   2> 3770670 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@14b1666d
   [junit4]   2> 3770671 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:45072/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 3770685 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:45072/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 3770686 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=hdfs://localhost:45072/solr_hdfs_home/control_collection/core_node1/data/index/
   [junit4]   2> 3770696 WARN  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory 'hdfs:/localhost:45072/solr_hdfs_home/control_collection/core_node1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 3770696 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:45072/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 3770704 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
   [junit4]   2> 3770704 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
   [junit4]   2> 3770707 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
   [junit4]   2> 3770707 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:45072/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 3770708 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=39, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
   [junit4]   2> 3770837 INFO  (IPC Server handler 1 on 45072) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:56716 is added to blk_1073741825_1001{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-ac0a84a6-bb67-44fb-a723-4ccf84b541fc:NORMAL:127.0.0.1:58382|RBW], ReplicaUnderConstruction[[DISK]DS-19b96dcd-eb77-4930-b2a3-8e208cb3efbc:NORMAL:127.0.0.1:56716|FINALIZED]]} size 0
   [junit4]   2> 3770841 INFO  (IPC Server handler 8 on 45072) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:58382 is added to blk_1073741825_1001 size 71
   [junit4]   2> 3770842 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(BlockDirectory(HdfsDirectory@b765ba46 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@3b33e37d); maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 3770842 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 3770849 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 3770849 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 3770849 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 3770849 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 3770849 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 3770850 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 3770850 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 3770850 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 3770850 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 3770850 WARN  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
   [junit4]   2> 3770853 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 3770853 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in hdfs://localhost:45072/solr_hdfs_home/control_collection/core_node1/data/
   [junit4]   2> 3770854 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 3770855 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 3770856 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 3770860 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /update/csv,/schema,/get,standard,/replication,/update/json,/admin/system,/admin/luke,/admin/threads,/update,/admin/logging,/dump,/admin/properties,/admin/ping,/admin/file,/admin/mbeans,/admin/plugins,/admin/segments,/update/json/docs,/config
   [junit4]   2> 3770861 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 3770862 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
   [junit4]   2> 3770862 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing HdfsUpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 tlogDfsReplication=2
   [junit4]   2> 3770873 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 3770873 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 3770877 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=37, maxMergeAtOnceExplicit=14, maxMergedSegmentMB=33.234375, floorSegmentMB=1.8076171875, forceMergeDeletesPctAllowed=17.689197199279555, segmentsPerTier=12.0, maxCFSSegmentSizeMB=1.9453125, noCFSRatio=0.0
   [junit4]   2> 3770894 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(BlockDirectory(HdfsDirectory@b765ba46 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@3b33e37d); maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 3770894 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 3770894 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@3ab1ee4b[collection1] main
   [junit4]   2> 3770894 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 3770895 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 3770895 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 3770896 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 3770896 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 3770896 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 3770897 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 3770897 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 3770897 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 3770897 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 3770897 INFO  (searcherExecutor-5955-thread-1-processing-x:collection1 s:shard1 n:127.0.0.1:50813_jv_rk c:control_collection r:core_node1) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@3ab1ee4b[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 3770898 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 3770898 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 256 version buckets from index
   [junit4]   2> 3770898 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 3770898 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1518976628010516480
   [junit4]   2> 3770898 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 0.0ms to seed version buckets with highest version 1518976628010516480
   [junit4]   2> 3770898 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 3770899 INFO  (coreLoadExecutor-5954-thread-1-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 3770907 INFO  (coreZkRegister-5948-thread-1-processing-x:collection1 s:shard1 n:127.0.0.1:50813_jv_rk c:control_collection r:core_node1) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:50813/jv_rk collection:control_collection shard:shard1
   [junit4]   2> 3770908 INFO  (coreZkRegister-5948-thread-1-processing-x:collection1 s:shard1 n:127.0.0.1:50813_jv_rk c:control_collection r:core_node1) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 3770911 INFO  (coreZkRegister-5948-thread-1-processing-x:collection1 s:shard1 n:127.0.0.1:50813_jv_rk c:control_collection r:core_node1) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94936039073054724-core_node1-n_0000000000
   [junit4]   2> 3770911 INFO  (coreZkRegister-5948-thread-1-processing-x:collection1 s:shard1 n:127.0.0.1:50813_jv_rk c:control_collection r:core_node1) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 3770913 INFO  (coreZkRegister-5948-thread-1-processing-x:collection1 s:shard1 n:127.0.0.1:50813_jv_rk c:control_collection r:core_node1) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 3770913 INFO  (coreZkRegister-5948-thread-1-processing-x:collection1 s:shard1 n:127.0.0.1:50813_jv_rk c:control_collection r:core_node1) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 3770913 INFO  (OverseerStateUpdate-94936039073054724-127.0.0.1:50813_jv_rk-n_0000000000) [n:127.0.0.1:50813_jv_rk    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 3770913 INFO  (coreZkRegister-5948-thread-1-processing-x:collection1 s:shard1 n:127.0.0.1:50813_jv_rk c:control_collection r:core_node1) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:50813/jv_rk/collection1/
   [junit4]   2> 3770913 INFO  (coreZkRegister-5948-thread-1-processing-x:collection1 s:shard1 n:127.0.0.1:50813_jv_rk c:control_collection r:core_node1) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 3770914 INFO  (zkCallback-1631-thread-3-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3770914 INFO  (zkCallback-1631-thread-2-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3770917 INFO  (coreZkRegister-5948-thread-1-processing-x:collection1 s:shard1 n:127.0.0.1:50813_jv_rk c:control_collection r:core_node1) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:50813/jv_rk/collection1/ has no replicas
   [junit4]   2> 3770919 INFO  (coreZkRegister-5948-thread-1-processing-x:collection1 s:shard1 n:127.0.0.1:50813_jv_rk c:control_collection r:core_node1) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 3770920 INFO  (coreZkRegister-5948-thread-1-processing-x:collection1 s:shard1 n:127.0.0.1:50813_jv_rk c:control_collection r:core_node1) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 3770924 INFO  (OverseerStateUpdate-94936039073054724-127.0.0.1:50813_jv_rk-n_0000000000) [n:127.0.0.1:50813_jv_rk    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:50813/jv_rk",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 2
   [junit4]   2> 3770925 INFO  (coreZkRegister-5948-thread-1-processing-x:collection1 s:shard1 n:127.0.0.1:50813_jv_rk c:control_collection r:core_node1) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:50813/jv_rk/collection1/ shard1
   [junit4]   2> 3771026 INFO  (zkCallback-1631-thread-2-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3771026 INFO  (zkCallback-1631-thread-3-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3771067 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3771076 INFO  (coreZkRegister-5948-thread-1-processing-x:collection1 s:shard1 n:127.0.0.1:50813_jv_rk c:control_collection r:core_node1) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:50813/jv_rk/collection1/ and leader is http://127.0.0.1:50813/jv_rk/collection1/
   [junit4]   2> 3771076 INFO  (coreZkRegister-5948-thread-1-processing-x:collection1 s:shard1 n:127.0.0.1:50813_jv_rk c:control_collection r:core_node1) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:50813/jv_rk
   [junit4]   2> 3771076 INFO  (coreZkRegister-5948-thread-1-processing-x:collection1 s:shard1 n:127.0.0.1:50813_jv_rk c:control_collection r:core_node1) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 3771076 INFO  (coreZkRegister-5948-thread-1-processing-x:collection1 s:shard1 n:127.0.0.1:50813_jv_rk c:control_collection r:core_node1) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 3771076 INFO  (coreZkRegister-5948-thread-1-processing-x:collection1 s:shard1 n:127.0.0.1:50813_jv_rk c:control_collection r:core_node1) [n:127.0.0.1:50813_jv_rk c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 3771079 INFO  (OverseerStateUpdate-94936039073054724-127.0.0.1:50813_jv_rk-n_0000000000) [n:127.0.0.1:50813_jv_rk    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "state":"active",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "node_name":"127.0.0.1:50813_jv_rk",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:50813/jv_rk",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "collection":"control_collection"} current state version: 3
   [junit4]   2> 3771079 INFO  (OverseerStateUpdate-94936039073054724-127.0.0.1:50813_jv_rk-n_0000000000) [n:127.0.0.1:50813_jv_rk    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "state":"active",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "node_name":"127.0.0.1:50813_jv_rk",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:50813/jv_rk",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "collection":"control_collection"}
   [junit4]   2> 3771159 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3771169 INFO  (zkCallback-1633-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7115e362 name:ZooKeeperConnection Watcher:127.0.0.1:49674/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3771169 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3771169 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3771170 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 3771171 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 3771182 INFO  (zkCallback-1633-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3771182 INFO  (zkCallback-1631-thread-2-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3771182 INFO  (zkCallback-1631-thread-3-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3771667 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsBasicDistributedZkTest_AF61550CE06C52F5-001/shard-1-001/cores/collection1
   [junit4]   2> 3771668 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsBasicDistributedZkTest_AF61550CE06C52F5-001/shard-1-001
   [junit4]   2> 3771681 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 3771766 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@4d11a483{/jv_rk,null,AVAILABLE}
   [junit4]   2> 3771766 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.e.j.s.ServerConnector Started ServerConnector@4c86bedf{HTTP/1.1}{127.0.0.1:41067}
   [junit4]   2> 3771766 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.e.j.s.Server Started @3774574ms
   [junit4]   2> 3771767 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsBasicDistributedZkTest_AF61550CE06C52F5-001/shard-1-001/cores, solr.data.dir=hdfs://localhost:45072/hdfs__localhost_45072__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-5.x_solr_build_solr-core_test_J0_temp_solr.cloud.hdfs.HdfsBasicDistributedZkTest_AF61550CE06C52F5-001_tempDir-002_jetty1, hostContext=/jv_rk, hostPort=41067, solrconfig=solrconfig.xml}
   [junit4]   2> 3771767 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@3827de24
   [junit4]   2> 3771767 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsBasicDistributedZkTest_AF61550CE06C52F5-001/shard-1-001'
   [junit4]   2> 3771767 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 3771767 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 3771767 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3771778 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3771793 INFO  (zkCallback-1634-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@58f690fb name:ZooKeeperConnection Watcher:127.0.0.1:49674/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3771793 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3771794 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3771795 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 3771795 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsBasicDistributedZkTest_AF61550CE06C52F5-001/shard-1-001/solr.xml
   [junit4]   2> 3771814 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsBasicDistributedZkTest_AF61550CE06C52F5-001/shard-1-001/cores
   [junit4]   2> 3771814 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.CoreContainer New CoreContainer 1906471651
   [junit4]   2> 3771814 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsBasicDistributedZkTest_AF61550CE06C52F5-001/shard-1-001]
   [junit4]   2> 3771814 WARN  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsBasicDistributedZkTest_AF61550CE06C52F5-001/shard-1-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsBasicDistributedZkTest_AF61550CE06C52F5-001/shard-1-001/lib
   [junit4]   2> 3771823 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 3771824 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 3771824 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 3771824 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 3771824 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:49674/solr
   [junit4]   2> 3771824 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 3771824 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3771837 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3771848 INFO  (zkCallback-1636-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2a28e6cf name:ZooKeeperConnection Watcher:127.0.0.1:49674 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3771850 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3771850 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3771866 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [n:127.0.0.1:41067_jv_rk    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3771975 INFO  (zkCallback-1637-thread-1-processing-n:127.0.0.1:41067_jv_rk) [n:127.0.0.1:41067_jv_rk    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5257f9e3 name:ZooKeeperConnection Watcher:127.0.0.1:49674/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3771975 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [n:127.0.0.1:41067_jv_rk    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3771981 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [n:127.0.0.1:41067_jv_rk    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 3773002 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [n:127.0.0.1:41067_jv_rk    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:41067_jv_rk
   [junit4]   2> 3773002 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [n:127.0.0.1:41067_jv_rk    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:41067_jv_rk
   [junit4]   2> 3773003 INFO  (zkCallback-1633-thread-1) [    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3773003 INFO  (zkCallback-1631-thread-3-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3773003 INFO  (zkCallback-1631-thread-2-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3773004 INFO  (zkCallback-1637-thread-1-processing-n:127.0.0.1:41067_jv_rk) [n:127.0.0.1:41067_jv_rk    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3773005 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [n:127.0.0.1:41067_jv_rk    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 3773005 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [n:127.0.0.1:41067_jv_rk    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94936039073054728-127.0.0.1:41067_jv_rk-n_0000000001
   [junit4]   2> 3773006 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [n:127.0.0.1:41067_jv_rk    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/94936039073054724-127.0.0.1:50813_jv_rk-n_0000000000 to know if I could be the leader
   [junit4]   2> 3773014 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [n:127.0.0.1:41067_jv_rk    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 3773014 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [n:127.0.0.1:41067_jv_rk    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 3773017 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [n:127.0.0.1:41067_jv_rk    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsBasicDistributedZkTest_AF61550CE06C52F5-001/shard-1-001/cores
   [junit4]   2> 3773018 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [n:127.0.0.1:41067_jv_rk    ] o.a.s.c.SolrCore Created CoreDescriptor: {coreNodeName=, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsBasicDistributedZkTest_AF61550CE06C52F5-001/shard-1-001/cores/collection1, collection=collection1, loadOnStartup=true, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsBasicDistributedZkTest_AF61550CE06C52F5-001/shard-1-001/cores/collection1/, dataDir=data/, config=solrconfig.xml, configSetProperties=configsetprops.json, name=collection1, transient=false, schema=schema.xml, shard=}
   [junit4]   2> 3773018 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [n:127.0.0.1:41067_jv_rk    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsBasicDistributedZkTest_AF61550CE06C52F5-001/shard-1-001/cores/collection1/
   [junit4]   2> 3773018 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AF61550CE06C52F5]) [n:127.0.0.1:41067_jv_rk    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 3773034 INFO  (coreLoadExecutor-5965-thread-1-processing-n:127.0.0.1:41067_jv_rk) [n:127.0.0.1:41067_jv_rk c:collection1   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 3773034 INFO  (coreLoadExecutor-5965-thread-1-processing-n:127.0.0.1:41067_jv_rk) [n:127.0.0.1:41067_jv_rk c:collection1   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 3773036 INFO  (OverseerStateUpdate-94936039073054724-127.0.0.1:50813_jv_rk-n_0000000000) [n:127.0.0.1:50813_jv_rk    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "state":"down",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "node_name":"127.0.0.1:41067_jv_rk",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:41067/jv_rk",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "collection":"collection1"} current state version: 4
   [junit4]   2> 3773036 INFO  (OverseerStateUpdate-94936039073054724-127.0.0.1:50813_jv_rk-n_0000000000) [n:127.0.0.1:50813_jv_rk    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "state":"down",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "node_name":"127.0.0.1:41067_jv_rk",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:41067/jv_rk",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "collection":"collection1"}
   [junit4]   2> 3773036 INFO  (OverseerStateUpdate-94936039073054724-127.0.0.1:50813_jv_rk-n_0000000000) [n:127.0.0.1:50813_jv_rk    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 3773036 INFO  (OverseerStateUpdate-94936039073054724-127.0.0.1:50813_jv_rk-n_0000000000) [n:127.0.0.1:50813_jv_rk    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard2
   [junit4]   2> 3773037 INFO  (zkCallback-1633-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 3773038 INFO  (zkCallback-1631-thread-2-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 3773038 INFO  (zkCallback-1631-thread-3-processing-n:127.0.0.1:50813_jv_rk) [n:127.0.0.1:50813_jv_rk    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 3773039 INFO  (zkCallback-1637-thread-1-processing-n:127.0.0.1:41067_jv_rk) [n:127.0.0.1:41067_jv_rk    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - upda

[...truncated too long message...]

in-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-authn-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-authz-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-changelog-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-collective-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-event-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-exception-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-journal-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-normalization-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-operational-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-referral-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-schema-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-subtree-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-trigger-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-jdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-jdbm1-2.0.0-M2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-kerberos-codec-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-ldif-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-mavibot-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-protocol-kerberos-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-protocol-ldap-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-protocol-shared-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-xdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/api-all-1.0.0-M20.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/bcprov-jdk15-1.45.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/cglib-nodep-2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/commons-collections-3.2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/commons-math3-3.4.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/easymock-3.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/ehcache-core-2.4.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/hadoop-common-2.6.0-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/hadoop-hdfs-2.6.0-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/hadoop-minikdc-2.6.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jackson-annotations-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jackson-databind-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jcl-over-slf4j-1.7.7.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jersey-core-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jersey-server-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jetty-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jetty-util-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/mina-core-2.0.0-M5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/objenesis-1.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/analysis/icu/lucene-analyzers-icu-5.5.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-analysis-extras/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/contrib/analysis-extras/lib/icu4j-54.1.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-launcher.jar:/x1/jenkins/.ant/lib/ivy-2.3.0.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-antlr.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-regexp.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-resolver.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-net.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jmf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-oro.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-xalan2.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-logging.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jsch.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit4.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-javamail.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jai.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jdepend.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bsf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-swing.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-netrexx.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bcel.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-testutil.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-log4j.jar:/x1/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.7/lib/tools.jar:/x1/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.3.1.jar com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J1-20151127_062023_552.events @/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J1-20151127_062023_552.suites -stdin
   [junit4] ERROR: JVM J1 ended with an exception: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1503)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:130)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:963)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:960)
   [junit4] 	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4] 	at java.lang.Thread.run(Thread.java:745)

BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: At least one slave process threw an exception, first: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.

Total time: 211 minutes 38 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1028 - Still Failing

[Apache Jenkins Server <je...@builds.apache.org>]

Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1028/

3 tests failed.
FAILED:  org.apache.solr.hadoop.MorphlineBasicMiniMRTest.mrRun

Error Message:
Failed on local exception: java.io.IOException: Connection reset by peer; Host Details : local host is: "lucene1-us-west/10.41.0.5"; destination host is: "lucene1-us-west.apache.org":60346; 

Stack Trace:
java.io.IOException: Failed on local exception: java.io.IOException: Connection reset by peer; Host Details : local host is: "lucene1-us-west/10.41.0.5"; destination host is: "lucene1-us-west.apache.org":60346; 
	at __randomizedtesting.SeedInfo.seed([A8505A62BBC97604:A602EE6CBA5F440B]:0)
	at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:772)
	at org.apache.hadoop.ipc.Client.call(Client.java:1472)
	at org.apache.hadoop.ipc.Client.call(Client.java:1399)
	at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:232)
	at com.sun.proxy.$Proxy109.getClusterMetrics(Unknown Source)
	at org.apache.hadoop.yarn.api.impl.pb.client.ApplicationClientProtocolPBClientImpl.getClusterMetrics(ApplicationClientProtocolPBClientImpl.java:202)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:187)
	at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)
	at com.sun.proxy.$Proxy110.getClusterMetrics(Unknown Source)
	at org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.getYarnClusterMetrics(YarnClientImpl.java:461)
	at org.apache.hadoop.mapred.ResourceMgrDelegate.getClusterMetrics(ResourceMgrDelegate.java:151)
	at org.apache.hadoop.mapred.YARNRunner.getClusterMetrics(YARNRunner.java:179)
	at org.apache.hadoop.mapreduce.Cluster.getClusterStatus(Cluster.java:246)
	at org.apache.hadoop.mapred.JobClient$3.run(JobClient.java:719)
	at org.apache.hadoop.mapred.JobClient$3.run(JobClient.java:717)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
	at org.apache.hadoop.mapred.JobClient.getClusterStatus(JobClient.java:717)
	at org.apache.solr.hadoop.MapReduceIndexerTool.run(MapReduceIndexerTool.java:644)
	at org.apache.solr.hadoop.MapReduceIndexerTool.run(MapReduceIndexerTool.java:607)
	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
	at org.apache.solr.hadoop.MorphlineBasicMiniMRTest.mrRun(MorphlineBasicMiniMRTest.java:363)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.IOException: Connection reset by peer
	at sun.nio.ch.FileDispatcherImpl.read0(Native Method)
	at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
	at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223)
	at sun.nio.ch.IOUtil.read(IOUtil.java:197)
	at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:384)
	at org.apache.hadoop.net.SocketInputStream$Reader.performIO(SocketInputStream.java:57)
	at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:142)
	at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161)
	at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131)
	at java.io.FilterInputStream.read(FilterInputStream.java:133)
	at java.io.FilterInputStream.read(FilterInputStream.java:133)
	at org.apache.hadoop.ipc.Client$Connection$PingInputStream.read(Client.java:513)
	at java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
	at java.io.BufferedInputStream.read(BufferedInputStream.java:254)
	at java.io.DataInputStream.readInt(DataInputStream.java:387)
	at org.apache.hadoop.ipc.Client$Connection.receiveRpcResponse(Client.java:1071)
	at org.apache.hadoop.ipc.Client$Connection.run(Client.java:966)


FAILED:  junit.framework.TestSuite.org.apache.solr.hadoop.MorphlineBasicMiniMRTest

Error Message:
1 thread leaked from SUITE scope at org.apache.solr.hadoop.MorphlineBasicMiniMRTest:     1) Thread[id=469, name=Node Status Updater, state=TIMED_WAITING, group=TGRP-MorphlineBasicMiniMRTest]         at java.lang.Object.wait(Native Method)         at org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl$1.run(NodeStatusUpdaterImpl.java:659)         at java.lang.Thread.run(Thread.java:745)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.hadoop.MorphlineBasicMiniMRTest: 
   1) Thread[id=469, name=Node Status Updater, state=TIMED_WAITING, group=TGRP-MorphlineBasicMiniMRTest]
        at java.lang.Object.wait(Native Method)
        at org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl$1.run(NodeStatusUpdaterImpl.java:659)
        at java.lang.Thread.run(Thread.java:745)
	at __randomizedtesting.SeedInfo.seed([A8505A62BBC97604]:0)


FAILED:  org.apache.solr.hadoop.MorphlineGoLiveMiniMRTest.test

Error Message:
Failed on local exception: java.io.IOException: Connection reset by peer; Host Details : local host is: "lucene1-us-west/10.41.0.5"; destination host is: "lucene1-us-west.apache.org":42452; 

Stack Trace:
java.io.IOException: Failed on local exception: java.io.IOException: Connection reset by peer; Host Details : local host is: "lucene1-us-west/10.41.0.5"; destination host is: "lucene1-us-west.apache.org":42452; 
	at __randomizedtesting.SeedInfo.seed([A8505A62BBC97604:200465B815351BFC]:0)
	at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:772)
	at org.apache.hadoop.ipc.Client.call(Client.java:1472)
	at org.apache.hadoop.ipc.Client.call(Client.java:1399)
	at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:232)
	at com.sun.proxy.$Proxy111.getClusterMetrics(Unknown Source)
	at org.apache.hadoop.yarn.api.impl.pb.client.ApplicationClientProtocolPBClientImpl.getClusterMetrics(ApplicationClientProtocolPBClientImpl.java:202)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:187)
	at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)
	at com.sun.proxy.$Proxy112.getClusterMetrics(Unknown Source)
	at org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.getYarnClusterMetrics(YarnClientImpl.java:461)
	at org.apache.hadoop.mapred.ResourceMgrDelegate.getClusterMetrics(ResourceMgrDelegate.java:151)
	at org.apache.hadoop.mapred.YARNRunner.getClusterMetrics(YARNRunner.java:179)
	at org.apache.hadoop.mapreduce.Cluster.getClusterStatus(Cluster.java:246)
	at org.apache.hadoop.mapred.JobClient$3.run(JobClient.java:719)
	at org.apache.hadoop.mapred.JobClient$3.run(JobClient.java:717)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
	at org.apache.hadoop.mapred.JobClient.getClusterStatus(JobClient.java:717)
	at org.apache.solr.hadoop.MapReduceIndexerTool.run(MapReduceIndexerTool.java:644)
	at org.apache.solr.hadoop.MapReduceIndexerTool.run(MapReduceIndexerTool.java:607)
	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
	at org.apache.solr.hadoop.MorphlineGoLiveMiniMRTest.test(MorphlineGoLiveMiniMRTest.java:402)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:963)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:938)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.IOException: Connection reset by peer
	at sun.nio.ch.FileDispatcherImpl.read0(Native Method)
	at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
	at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223)
	at sun.nio.ch.IOUtil.read(IOUtil.java:197)
	at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:384)
	at org.apache.hadoop.net.SocketInputStream$Reader.performIO(SocketInputStream.java:57)
	at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:142)
	at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161)
	at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131)
	at java.io.FilterInputStream.read(FilterInputStream.java:133)
	at java.io.FilterInputStream.read(FilterInputStream.java:133)
	at org.apache.hadoop.ipc.Client$Connection$PingInputStream.read(Client.java:513)
	at java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
	at java.io.BufferedInputStream.read(BufferedInputStream.java:254)
	at java.io.DataInputStream.readInt(DataInputStream.java:387)
	at org.apache.hadoop.ipc.Client$Connection.receiveRpcResponse(Client.java:1071)
	at org.apache.hadoop.ipc.Client$Connection.run(Client.java:966)




Build Log:
[...truncated 18080 lines...]
   [junit4] Suite: org.apache.solr.hadoop.MorphlineBasicMiniMRTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/init-core-data-001
   [junit4]   2> 6901 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true)
   [junit4]   2> 7064 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.c.C.deprecation dfs.permissions is deprecated. Instead, use dfs.permissions.enabled
   [junit4]   1> Formatting using clusterid: testClusterID
   [junit4]   2> 7408 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.n.FSNamesystem No KeyProvider found.
   [junit4]   2> 7447 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.n.FSNamesystem fsLock is fair:true
   [junit4]   2> 7534 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.c.C.deprecation hadoop.configured.node.mapping is deprecated. Instead, use net.topology.configured.node.mapping
   [junit4]   2> 7534 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.b.DatanodeManager dfs.block.invalidate.limit=1000
   [junit4]   2> 7535 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.b.DatanodeManager dfs.namenode.datanode.registration.ip-hostname-check=true
   [junit4]   2> 7538 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.b.BlockManager dfs.namenode.startup.delay.block.deletion.sec is set to 000:00:00:00.000
   [junit4]   2> 7541 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.b.BlockManager The block deletion will start around 2015 nov 26 06:30:46
   [junit4]   2> 7546 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.u.GSet Computing capacity for map BlocksMap
   [junit4]   2> 7546 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.u.GSet VM type       = 64-bit
   [junit4]   2> 7549 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.u.GSet 2.0% max memory 455 MB = 9.1 MB
   [junit4]   2> 7549 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.u.GSet capacity      = 2^20 = 1048576 entries
   [junit4]   2> 7582 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.b.BlockManager dfs.block.access.token.enable=false
   [junit4]   2> 7582 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.b.BlockManager defaultReplication         = 2
   [junit4]   2> 7582 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.b.BlockManager maxReplication             = 512
   [junit4]   2> 7582 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.b.BlockManager minReplication             = 1
   [junit4]   2> 7583 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.b.BlockManager maxReplicationStreams      = 2
   [junit4]   2> 7583 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.b.BlockManager shouldCheckForEnoughRacks  = false
   [junit4]   2> 7583 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.b.BlockManager replicationRecheckInterval = 3000
   [junit4]   2> 7583 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.b.BlockManager encryptDataTransfer        = false
   [junit4]   2> 7583 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.b.BlockManager maxNumBlocksToLog          = 1000
   [junit4]   2> 7585 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.n.FSNamesystem fsOwner             = jenkins (auth:SIMPLE)
   [junit4]   2> 7585 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.n.FSNamesystem supergroup          = supergroup
   [junit4]   2> 7585 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.n.FSNamesystem isPermissionEnabled = true
   [junit4]   2> 7586 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.n.FSNamesystem HA Enabled: false
   [junit4]   2> 7591 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.n.FSNamesystem Append Enabled: true
   [junit4]   2> 7850 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.u.GSet Computing capacity for map INodeMap
   [junit4]   2> 7850 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.u.GSet VM type       = 64-bit
   [junit4]   2> 7850 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.u.GSet 1.0% max memory 455 MB = 4.5 MB
   [junit4]   2> 7850 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.u.GSet capacity      = 2^19 = 524288 entries
   [junit4]   2> 7851 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.n.NameNode Caching file names occuring more than 10 times
   [junit4]   2> 7865 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.u.GSet Computing capacity for map cachedBlocks
   [junit4]   2> 7866 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.u.GSet VM type       = 64-bit
   [junit4]   2> 7866 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.u.GSet 0.25% max memory 455 MB = 1.1 MB
   [junit4]   2> 7866 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.u.GSet capacity      = 2^17 = 131072 entries
   [junit4]   2> 7868 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.n.FSNamesystem dfs.namenode.safemode.threshold-pct = 0.9990000128746033
   [junit4]   2> 7868 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.n.FSNamesystem dfs.namenode.safemode.min.datanodes = 0
   [junit4]   2> 7869 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.n.FSNamesystem dfs.namenode.safemode.extension     = 0
   [junit4]   2> 7872 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.n.FSNamesystem Retry cache on namenode is enabled
   [junit4]   2> 7872 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.n.FSNamesystem Retry cache will use 0.03 of total heap and retry cache entry expiry time is 600000 millis
   [junit4]   2> 7878 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.u.GSet Computing capacity for map NameNodeRetryCache
   [junit4]   2> 7878 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.u.GSet VM type       = 64-bit
   [junit4]   2> 7879 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.u.GSet 0.029999999329447746% max memory 455 MB = 139.8 KB
   [junit4]   2> 7879 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.u.GSet capacity      = 2^14 = 16384 entries
   [junit4]   2> 7890 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.n.NNConf ACLs enabled? false
   [junit4]   2> 7890 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.n.NNConf XAttrs enabled? true
   [junit4]   2> 7890 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.n.NNConf Maximum size of an xattr: 16384
   [junit4]   2> 7990 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.n.FSImage Allocated new BlockPoolId: BP-1956537108-10.41.0.5-1448515846412
   [junit4]   2> 8006 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.c.Storage Storage directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/name1 has been successfully formatted.
   [junit4]   2> 8011 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.c.Storage Storage directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/name2 has been successfully formatted.
   [junit4]   2> 8237 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.n.NNStorageRetentionManager Going to retain 1 images with txid >= 0
   [junit4]   2> 8240 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.n.NameNode createNameNode []
   [junit4]   2> 8297 WARN  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
   [junit4]   2> 8403 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.m.i.MetricsSystemImpl Scheduled snapshot period at 10 second(s).
   [junit4]   2> 8403 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.m.i.MetricsSystemImpl NameNode metrics system started
   [junit4]   2> 8406 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.n.NameNode fs.defaultFS is hdfs://127.0.0.1:0
   [junit4]   2> 8479 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.DFSUtil Starting Web-server for hdfs at: http://localhost:0
   [junit4]   2> 8619 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.m.log Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
   [junit4]   2> 8634 WARN  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 8666 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.HttpServer2 Added global filter 'safety' (class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter)
   [junit4]   2> 8671 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.HttpServer2 Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context hdfs
   [junit4]   2> 8671 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.HttpServer2 Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context static
   [junit4]   2> 8671 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.HttpServer2 Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs
   [junit4]   2> 8711 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.HttpServer2 Added filter 'org.apache.hadoop.hdfs.web.AuthFilter' (class=org.apache.hadoop.hdfs.web.AuthFilter)
   [junit4]   2> 8713 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.HttpServer2 addJerseyResourcePackage: packageName=org.apache.hadoop.hdfs.server.namenode.web.resources;org.apache.hadoop.hdfs.web.resources, pathSpec=/webhdfs/v1/*
   [junit4]   2> 8738 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.HttpServer2 Jetty bound to port 42072
   [junit4]   2> 8738 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 8820 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/hdfs to ./temp/Jetty_localhost_42072_hdfs____.xpjzih/webapp
   [junit4]   2> 9050 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 9923 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:42072
   [junit4]   2> 9935 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.n.FSNamesystem No KeyProvider found.
   [junit4]   2> 9935 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.n.FSNamesystem fsLock is fair:true
   [junit4]   2> 9936 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.b.DatanodeManager dfs.block.invalidate.limit=1000
   [junit4]   2> 9937 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.b.DatanodeManager dfs.namenode.datanode.registration.ip-hostname-check=true
   [junit4]   2> 9937 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.b.BlockManager dfs.namenode.startup.delay.block.deletion.sec is set to 000:00:00:00.000
   [junit4]   2> 9938 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.b.BlockManager The block deletion will start around 2015 nov 26 06:30:48
   [junit4]   2> 9938 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.u.GSet Computing capacity for map BlocksMap
   [junit4]   2> 9938 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.u.GSet VM type       = 64-bit
   [junit4]   2> 9939 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.u.GSet 2.0% max memory 455 MB = 9.1 MB
   [junit4]   2> 9943 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.u.GSet capacity      = 2^20 = 1048576 entries
   [junit4]   2> 9944 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.b.BlockManager dfs.block.access.token.enable=false
   [junit4]   2> 9944 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.b.BlockManager defaultReplication         = 2
   [junit4]   2> 9944 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.b.BlockManager maxReplication             = 512
   [junit4]   2> 9944 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.b.BlockManager minReplication             = 1
   [junit4]   2> 9944 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.b.BlockManager maxReplicationStreams      = 2
   [junit4]   2> 9945 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.b.BlockManager shouldCheckForEnoughRacks  = false
   [junit4]   2> 9945 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.b.BlockManager replicationRecheckInterval = 3000
   [junit4]   2> 9945 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.b.BlockManager encryptDataTransfer        = false
   [junit4]   2> 9945 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.b.BlockManager maxNumBlocksToLog          = 1000
   [junit4]   2> 9945 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.n.FSNamesystem fsOwner             = jenkins (auth:SIMPLE)
   [junit4]   2> 9946 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.n.FSNamesystem supergroup          = supergroup
   [junit4]   2> 9946 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.n.FSNamesystem isPermissionEnabled = true
   [junit4]   2> 9947 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.n.FSNamesystem HA Enabled: false
   [junit4]   2> 9947 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.n.FSNamesystem Append Enabled: true
   [junit4]   2> 9948 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.u.GSet Computing capacity for map INodeMap
   [junit4]   2> 9948 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.u.GSet VM type       = 64-bit
   [junit4]   2> 9948 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.u.GSet 1.0% max memory 455 MB = 4.5 MB
   [junit4]   2> 9948 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.u.GSet capacity      = 2^19 = 524288 entries
   [junit4]   2> 9949 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.n.NameNode Caching file names occuring more than 10 times
   [junit4]   2> 9949 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.u.GSet Computing capacity for map cachedBlocks
   [junit4]   2> 9949 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.u.GSet VM type       = 64-bit
   [junit4]   2> 9949 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.u.GSet 0.25% max memory 455 MB = 1.1 MB
   [junit4]   2> 9953 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.u.GSet capacity      = 2^17 = 131072 entries
   [junit4]   2> 9953 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.n.FSNamesystem dfs.namenode.safemode.threshold-pct = 0.9990000128746033
   [junit4]   2> 9954 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.n.FSNamesystem dfs.namenode.safemode.min.datanodes = 0
   [junit4]   2> 9954 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.n.FSNamesystem dfs.namenode.safemode.extension     = 0
   [junit4]   2> 9954 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.n.FSNamesystem Retry cache on namenode is enabled
   [junit4]   2> 9954 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.n.FSNamesystem Retry cache will use 0.03 of total heap and retry cache entry expiry time is 600000 millis
   [junit4]   2> 9954 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.u.GSet Computing capacity for map NameNodeRetryCache
   [junit4]   2> 9954 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.u.GSet VM type       = 64-bit
   [junit4]   2> 9955 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.u.GSet 0.029999999329447746% max memory 455 MB = 139.8 KB
   [junit4]   2> 9955 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.u.GSet capacity      = 2^14 = 16384 entries
   [junit4]   2> 9957 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.n.NNConf ACLs enabled? false
   [junit4]   2> 9957 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.n.NNConf XAttrs enabled? true
   [junit4]   2> 9957 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.n.NNConf Maximum size of an xattr: 16384
   [junit4]   2> 9962 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.c.Storage Lock on /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/name1/in_use.lock acquired by nodename 26303@lucene1-us-west
   [junit4]   2> 9968 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.c.Storage Lock on /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/name2/in_use.lock acquired by nodename 26303@lucene1-us-west
   [junit4]   2> 9976 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.n.FileJournalManager Recovering unfinalized segments in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/name1/current
   [junit4]   2> 9976 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.n.FileJournalManager Recovering unfinalized segments in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/name2/current
   [junit4]   2> 9988 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.n.FSImage No edit log streams selected.
   [junit4]   2> 10045 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.n.FSImageFormatPBINode Loading 1 INodes.
   [junit4]   2> 10065 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.n.FSImageFormatProtobuf Loaded FSImage in 0 seconds.
   [junit4]   2> 10066 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.n.FSImage Loaded image for txid 0 from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/name1/current/fsimage_0000000000000000000
   [junit4]   2> 10074 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.n.FSNamesystem Need to save fs image? false (staleImage=false, haEnabled=false, isRollingUpgrade=false)
   [junit4]   2> 10075 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.n.FSEditLog Starting log segment at 1
   [junit4]   2> 10156 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.n.NameCache initialized with 0 entries 0 lookups
   [junit4]   2> 10156 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.n.FSNamesystem Finished loading FSImage in 198 msecs
   [junit4]   2> 10617 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.n.NameNode RPC server is binding to localhost:0
   [junit4]   2> 10632 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.i.CallQueueManager Using callQueue class java.util.concurrent.LinkedBlockingQueue
   [junit4]   2> 10660 INFO  (Socket Reader #1 for port 55392) [    ] o.a.h.i.Server Starting Socket Reader #1 for port 55392
   [junit4]   2> 10723 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.n.NameNode Clients are to use localhost:55392 to access this namenode/service.
   [junit4]   2> 10738 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.n.FSNamesystem Registered FSNamesystemState MBean
   [junit4]   2> 10803 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.n.FSNamesystem Number of blocks under construction: 0
   [junit4]   2> 10803 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.n.FSNamesystem Number of blocks under construction: 0
   [junit4]   2> 10803 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.n.FSNamesystem initializing replication queues
   [junit4]   2> 10817 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.StateChange STATE* Leaving safe mode after 0 secs
   [junit4]   2> 10817 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.StateChange STATE* Network topology has 0 racks and 0 datanodes
   [junit4]   2> 10817 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.StateChange STATE* UnderReplicatedBlocks has 0 blocks
   [junit4]   2> 10831 INFO  (Replication Queue Initializer) [    ] o.a.h.h.s.b.BlockManager Total number of blocks            = 0
   [junit4]   2> 10833 INFO  (Replication Queue Initializer) [    ] o.a.h.h.s.b.BlockManager Number of invalid blocks          = 0
   [junit4]   2> 10833 INFO  (Replication Queue Initializer) [    ] o.a.h.h.s.b.BlockManager Number of under-replicated blocks = 0
   [junit4]   2> 10833 INFO  (Replication Queue Initializer) [    ] o.a.h.h.s.b.BlockManager Number of  over-replicated blocks = 0
   [junit4]   2> 10833 INFO  (Replication Queue Initializer) [    ] o.a.h.h.s.b.BlockManager Number of blocks being written    = 0
   [junit4]   2> 10834 INFO  (Replication Queue Initializer) [    ] o.a.h.h.StateChange STATE* Replication Queue initialization scan for invalid, over- and under-replicated blocks completed in 17 msec
   [junit4]   2> 10922 INFO  (IPC Server Responder) [    ] o.a.h.i.Server IPC Server Responder: starting
   [junit4]   2> 10924 INFO  (IPC Server listener on 55392) [    ] o.a.h.i.Server IPC Server listener on 55392: starting
   [junit4]   2> 10934 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.n.NameNode NameNode RPC up at: localhost/127.0.0.1:55392
   [junit4]   2> 10934 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.n.FSNamesystem Starting services required for active state
   [junit4]   2> 10947 INFO  (CacheReplicationMonitor(2137030917)) [    ] o.a.h.h.s.b.CacheReplicationMonitor Starting CacheReplicationMonitor with interval 30000 milliseconds
   [junit4]   2> 10947 INFO  (CacheReplicationMonitor(2137030917)) [    ] o.a.h.h.s.b.CacheReplicationMonitor Rescanning after 16976147130 milliseconds
   [junit4]   2> 10953 INFO  (CacheReplicationMonitor(2137030917)) [    ] o.a.h.h.s.b.CacheReplicationMonitor Scanned 0 directive(s) and 0 block(s) in 6 millisecond(s).
   [junit4]   2> 10959 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.MiniDFSCluster Starting DataNode 0 with dfs.datanode.data.dir: [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data1,[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data2
   [junit4]   2> 10998 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.m.i.MetricsSystemImpl DataNode metrics system started (again)
   [junit4]   2> 11000 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.d.DataNode Configured hostname is 127.0.0.1
   [junit4]   2> 11008 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.d.DataNode Starting DataNode with maxLockedMemory = 0
   [junit4]   2> 11017 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.d.DataNode Opened streaming server at /127.0.0.1:52183
   [junit4]   2> 11021 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.d.DataNode Balancing bandwith is 1048576 bytes/s
   [junit4]   2> 11021 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.d.DataNode Number threads for balancing is 5
   [junit4]   2> 11027 WARN  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 11028 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.HttpServer2 Added global filter 'safety' (class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter)
   [junit4]   2> 11029 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.HttpServer2 Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context datanode
   [junit4]   2> 11029 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.HttpServer2 Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context static
   [junit4]   2> 11029 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.HttpServer2 Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs
   [junit4]   2> 11036 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.HttpServer2 addJerseyResourcePackage: packageName=org.apache.hadoop.hdfs.server.datanode.web.resources;org.apache.hadoop.hdfs.web.resources, pathSpec=/webhdfs/v1/*
   [junit4]   2> 11036 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.HttpServer2 Jetty bound to port 58014
   [junit4]   2> 11036 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 11056 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_58014_datanode____b8giwn/webapp
   [junit4]   2> 11204 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 11517 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:58014
   [junit4]   2> 11519 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.d.DataNode dnUserName = jenkins
   [junit4]   2> 11520 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.d.DataNode supergroup = supergroup
   [junit4]   2> 11550 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.i.CallQueueManager Using callQueue class java.util.concurrent.LinkedBlockingQueue
   [junit4]   2> 11569 INFO  (Socket Reader #1 for port 49479) [    ] o.a.h.i.Server Starting Socket Reader #1 for port 49479
   [junit4]   2> 11581 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.d.DataNode Opened IPC server at /127.0.0.1:49479
   [junit4]   2> 11594 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.d.DataNode Refresh request received for nameservices: null
   [junit4]   2> 11598 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.d.DataNode Starting BPOfferServices for nameservices: <default>
   [junit4]   2> 11618 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:55392) [    ] o.a.h.h.s.d.DataNode Block pool <registering> (Datanode Uuid unassigned) service to localhost/127.0.0.1:55392 starting to offer service
   [junit4]   2> 11629 INFO  (IPC Server Responder) [    ] o.a.h.i.Server IPC Server Responder: starting
   [junit4]   2> 11630 INFO  (IPC Server listener on 49479) [    ] o.a.h.i.Server IPC Server listener on 49479: starting
   [junit4]   2> 11656 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.MiniDFSCluster Starting DataNode 1 with dfs.datanode.data.dir: [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data3,[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data4
   [junit4]   2> 11766 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.m.i.MetricsSystemImpl DataNode metrics system started (again)
   [junit4]   2> 11766 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.d.DataNode Configured hostname is 127.0.0.1
   [junit4]   2> 11766 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.d.DataNode Starting DataNode with maxLockedMemory = 0
   [junit4]   2> 11767 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.d.DataNode Opened streaming server at /127.0.0.1:47411
   [junit4]   2> 11767 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.d.DataNode Balancing bandwith is 1048576 bytes/s
   [junit4]   2> 11767 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.d.DataNode Number threads for balancing is 5
   [junit4]   2> 11769 WARN  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 11771 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.HttpServer2 Added global filter 'safety' (class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter)
   [junit4]   2> 11771 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.HttpServer2 Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context datanode
   [junit4]   2> 11771 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.HttpServer2 Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs
   [junit4]   2> 11771 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.HttpServer2 Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context static
   [junit4]   2> 11773 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.HttpServer2 addJerseyResourcePackage: packageName=org.apache.hadoop.hdfs.server.datanode.web.resources;org.apache.hadoop.hdfs.web.resources, pathSpec=/webhdfs/v1/*
   [junit4]   2> 11773 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.HttpServer2 Jetty bound to port 36053
   [junit4]   2> 11773 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 11830 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_36053_datanode____psqigi/webapp
   [junit4]   2> 11967 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 12304 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:55392) [    ] o.a.h.h.s.c.Storage DataNode version: -56 and NameNode layout version: -60
   [junit4]   2> 12318 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:55392) [    ] o.a.h.h.s.c.Storage Lock on /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data1/in_use.lock acquired by nodename 26303@lucene1-us-west
   [junit4]   2> 12320 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:55392) [    ] o.a.h.h.s.c.Storage Storage directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data1 is not formatted for BP-1956537108-10.41.0.5-1448515846412
   [junit4]   2> 12320 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:55392) [    ] o.a.h.h.s.c.Storage Formatting ...
   [junit4]   2> 12327 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:55392) [    ] o.a.h.h.s.c.Storage Lock on /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data2/in_use.lock acquired by nodename 26303@lucene1-us-west
   [junit4]   2> 12328 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:55392) [    ] o.a.h.h.s.c.Storage Storage directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data2 is not formatted for BP-1956537108-10.41.0.5-1448515846412
   [junit4]   2> 12328 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:55392) [    ] o.a.h.h.s.c.Storage Formatting ...
   [junit4]   2> 12339 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:36053
   [junit4]   2> 12340 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.d.DataNode dnUserName = jenkins
   [junit4]   2> 12340 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.d.DataNode supergroup = supergroup
   [junit4]   2> 12340 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.i.CallQueueManager Using callQueue class java.util.concurrent.LinkedBlockingQueue
   [junit4]   2> 12346 INFO  (Socket Reader #1 for port 54395) [    ] o.a.h.i.Server Starting Socket Reader #1 for port 54395
   [junit4]   2> 12351 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.d.DataNode Opened IPC server at /127.0.0.1:54395
   [junit4]   2> 12359 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.d.DataNode Refresh request received for nameservices: null
   [junit4]   2> 12359 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[A8505A62BBC97604]-worker) [    ] o.a.h.h.s.d.DataNode Starting BPOfferServices for nameservices: <default>
   [junit4]   2> 12361 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:55392) [    ] o.a.h.h.s.d.DataNode Block pool <registering> (Datanode Uuid unassigned) service to localhost/127.0.0.1:55392 starting to offer service
   [junit4]   2> 12365 INFO  (IPC Server Responder) [    ] o.a.h.i.Server IPC Server Responder: starting
   [junit4]   2> 12366 INFO  (IPC Server listener on 54395) [    ] o.a.h.i.Server IPC Server listener on 54395: starting
   [junit4]   2> 12387 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:55392) [    ] o.a.h.h.s.c.Storage DataNode version: -56 and NameNode layout version: -60
   [junit4]   2> 12393 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:55392) [    ] o.a.h.h.s.c.Storage Lock on /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data3/in_use.lock acquired by nodename 26303@lucene1-us-west
   [junit4]   2> 12394 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:55392) [    ] o.a.h.h.s.c.Storage Storage directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data3 is not formatted for BP-1956537108-10.41.0.5-1448515846412
   [junit4]   2> 12394 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:55392) [    ] o.a.h.h.s.c.Storage Formatting ...
   [junit4]   2> 12399 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:55392) [    ] o.a.h.h.s.c.Storage Lock on /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data4/in_use.lock acquired by nodename 26303@lucene1-us-west
   [junit4]   2> 12400 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:55392) [    ] o.a.h.h.s.c.Storage Storage directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data4 is not formatted for BP-1956537108-10.41.0.5-1448515846412
   [junit4]   2> 12400 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:55392) [    ] o.a.h.h.s.c.Storage Formatting ...
   [junit4]   2> 12534 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:55392) [    ] o.a.h.h.s.c.Storage Analyzing storage directories for bpid BP-1956537108-10.41.0.5-1448515846412
   [junit4]   2> 12535 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:55392) [    ] o.a.h.h.s.c.Storage Locking is disabled
   [junit4]   2> 12536 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:55392) [    ] o.a.h.h.s.c.Storage Storage directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data1/current/BP-1956537108-10.41.0.5-1448515846412 is not formatted.
   [junit4]   2> 12537 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:55392) [    ] o.a.h.h.s.c.Storage Formatting ...
   [junit4]   2> 12537 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:55392) [    ] o.a.h.h.s.c.Storage Formatting block pool BP-1956537108-10.41.0.5-1448515846412 directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data1/current/BP-1956537108-10.41.0.5-1448515846412/current
   [junit4]   2> 12542 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:55392) [    ] o.a.h.h.s.c.Storage Locking is disabled
   [junit4]   2> 12543 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:55392) [    ] o.a.h.h.s.c.Storage Storage directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data2/current/BP-1956537108-10.41.0.5-1448515846412 is not formatted.
   [junit4]   2> 12543 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:55392) [    ] o.a.h.h.s.c.Storage Formatting ...
   [junit4]   2> 12543 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:55392) [    ] o.a.h.h.s.c.Storage Formatting block pool BP-1956537108-10.41.0.5-1448515846412 directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data2/current/BP-1956537108-10.41.0.5-1448515846412/current
   [junit4]   2> 12546 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:55392) [    ] o.a.h.h.s.c.Storage Restored 0 block files from trash.
   [junit4]   2> 12546 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:55392) [    ] o.a.h.h.s.c.Storage Restored 0 block files from trash.
   [junit4]   2> 12548 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:55392) [    ] o.a.h.h.s.d.DataNode Setting up storage: nsid=1907285210;bpid=BP-1956537108-10.41.0.5-1448515846412;lv=-56;nsInfo=lv=-60;cid=testClusterID;nsid=1907285210;c=0;bpid=BP-1956537108-10.41.0.5-1448515846412;dnuuid=null
   [junit4]   2> 12552 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:55392) [    ] o.a.h.h.s.d.DataNode Generated and persisted new Datanode UUID 33118d58-6a16-4f9a-b990-610d64bf8614
   [junit4]   2> 12596 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:55392) [    ] o.a.h.h.s.c.Storage Analyzing storage directories for bpid BP-1956537108-10.41.0.5-1448515846412
   [junit4]   2> 12596 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:55392) [    ] o.a.h.h.s.c.Storage Locking is disabled
   [junit4]   2> 12597 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:55392) [    ] o.a.h.h.s.c.Storage Storage directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data3/current/BP-1956537108-10.41.0.5-1448515846412 is not formatted.
   [junit4]   2> 12597 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:55392) [    ] o.a.h.h.s.c.Storage Formatting ...
   [junit4]   2> 12597 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:55392) [    ] o.a.h.h.s.c.Storage Formatting block pool BP-1956537108-10.41.0.5-1448515846412 directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data3/current/BP-1956537108-10.41.0.5-1448515846412/current
   [junit4]   2> 12600 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:55392) [    ] o.a.h.h.s.c.Storage Locking is disabled
   [junit4]   2> 12601 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:55392) [    ] o.a.h.h.s.c.Storage Storage directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data4/current/BP-1956537108-10.41.0.5-1448515846412 is not formatted.
   [junit4]   2> 12601 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:55392) [    ] o.a.h.h.s.c.Storage Formatting ...
   [junit4]   2> 12601 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:55392) [    ] o.a.h.h.s.c.Storage Formatting block pool BP-1956537108-10.41.0.5-1448515846412 directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data4/current/BP-1956537108-10.41.0.5-1448515846412/current
   [junit4]   2> 12604 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:55392) [    ] o.a.h.h.s.c.Storage Restored 0 block files from trash.
   [junit4]   2> 12605 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:55392) [    ] o.a.h.h.s.c.Storage Restored 0 block files from trash.
   [junit4]   2> 12607 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:55392) [    ] o.a.h.h.s.d.DataNode Setting up storage: nsid=1907285210;bpid=BP-1956537108-10.41.0.5-1448515846412;lv=-56;nsInfo=lv=-60;cid=testClusterID;nsid=1907285210;c=0;bpid=BP-1956537108-10.41.0.5-1448515846412;dnuuid=null
   [junit4]   2> 12611 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:55392) [    ] o.a.h.h.s.d.DataNode Generated and persisted new Datanode UUID a6c625f4-dc0f-4707-bb26-9f39240219d6
   [junit4]   2> 12662 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:55392) [    ] o.a.h.h.s.d.f.i.FsDatasetImpl Added new volume: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data3/current
   [junit4]   2> 12665 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:55392) [    ] o.a.h.h.s.d.f.i.FsDatasetImpl Added volume - /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data3/current, StorageType: DISK
   [junit4]   2> 12665 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:55392) [    ] o.a.h.h.s.d.f.i.FsDatasetImpl Added new volume: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data4/current
   [junit4]   2> 12665 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:55392) [    ] o.a.h.h.s.d.f.i.FsDatasetImpl Added volume - /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data4/current, StorageType: DISK
   [junit4]   2> 12668 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:55392) [    ] o.a.h.h.s.d.f.i.FsDatasetImpl Added new volume: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data1/current
   [junit4]   2> 12685 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:55392) [    ] o.a.h.h.s.d.f.i.FsDatasetImpl Added volume - /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data1/current, StorageType: DISK
   [junit4]   2> 12686 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:55392) [    ] o.a.h.h.s.d.f.i.FsDatasetImpl Added new volume: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data2/current
   [junit4]   2> 12686 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:55392) [    ] o.a.h.h.s.d.f.i.FsDatasetImpl Added volume - /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hdfs/build/dfs/data/data2/current, StorageType: DISK
   [junit4]   2> 12725 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_A8505A62BBC97604-001/tempDir-002/hd

[...truncated too long message...]

.n.FSNamesystem Stopping services started for standby state
   [junit4]   2> 79431 INFO  (SUITE-MorphlineGoLiveMiniMRTest-seed#[A8505A62BBC97604]-worker) [n:127.0.0.1:37438_zb%2Fq c:collection1 s:shard2 r:core_node5 x:collection1] o.m.log Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0
   [junit4]   2> 79533 INFO  (SUITE-MorphlineGoLiveMiniMRTest-seed#[A8505A62BBC97604]-worker) [n:127.0.0.1:37438_zb%2Fq c:collection1 s:shard2 r:core_node5 x:collection1] o.a.h.m.i.MetricsSystemImpl Stopping JobHistoryServer metrics system...
   [junit4]   2> 79537 ERROR (SUITE-MorphlineGoLiveMiniMRTest-seed#[A8505A62BBC97604]-worker) [n:127.0.0.1:37438_zb%2Fq c:collection1 s:shard2 r:core_node5 x:collection1] o.a.h.m.l.MethodMetric Error invoking method getBlocksTotal
   [junit4]   2> java.lang.reflect.InvocationTargetException
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   [junit4]   2> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2> 	at java.lang.reflect.Method.invoke(Method.java:606)
   [junit4]   2> 	at org.apache.hadoop.metrics2.lib.MethodMetric$2.snapshot(MethodMetric.java:111)
   [junit4]   2> 	at org.apache.hadoop.metrics2.lib.MethodMetric.snapshot(MethodMetric.java:144)
   [junit4]   2> 	at org.apache.hadoop.metrics2.lib.MetricsRegistry.snapshot(MetricsRegistry.java:387)
   [junit4]   2> 	at org.apache.hadoop.metrics2.lib.MetricsSourceBuilder$1.getMetrics(MetricsSourceBuilder.java:79)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.getMetrics(MetricsSourceAdapter.java:195)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.updateJmxCache(MetricsSourceAdapter.java:172)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.getMBeanInfo(MetricsSourceAdapter.java:151)
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getClassName(DefaultMBeanServerInterceptor.java:1804)
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.safeGetClassName(DefaultMBeanServerInterceptor.java:1595)
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.checkMBeanPermission(DefaultMBeanServerInterceptor.java:1813)
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.exclusiveUnregisterMBean(DefaultMBeanServerInterceptor.java:430)
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.unregisterMBean(DefaultMBeanServerInterceptor.java:415)
   [junit4]   2> 	at com.sun.jmx.mbeanserver.JmxMBeanServer.unregisterMBean(JmxMBeanServer.java:546)
   [junit4]   2> 	at org.apache.hadoop.metrics2.util.MBeans.unregister(MBeans.java:81)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.stopMBeans(MetricsSourceAdapter.java:227)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.stop(MetricsSourceAdapter.java:212)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.stopSources(MetricsSystemImpl.java:461)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.stop(MetricsSystemImpl.java:212)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.shutdown(MetricsSystemImpl.java:592)
   [junit4]   2> 	at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.shutdownInstance(DefaultMetricsSystem.java:72)
   [junit4]   2> 	at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.shutdown(DefaultMetricsSystem.java:68)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.metrics.NameNodeMetrics.shutdown(NameNodeMetrics.java:145)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.NameNode.stop(NameNode.java:822)
   [junit4]   2> 	at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1720)
   [junit4]   2> 	at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1699)
   [junit4]   2> 	at org.apache.solr.hadoop.MorphlineGoLiveMiniMRTest.teardownClass(MorphlineGoLiveMiniMRTest.java:236)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   [junit4]   2> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2> 	at java.lang.reflect.Method.invoke(Method.java:606)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:834)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> Caused by: java.lang.NullPointerException
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.blockmanagement.BlocksMap.size(BlocksMap.java:198)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.getTotalBlocks(BlockManager.java:3291)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlocksTotal(FSNamesystem.java:6223)
   [junit4]   2> 	... 53 more
   [junit4]   2> 79551 INFO  (SUITE-MorphlineGoLiveMiniMRTest-seed#[A8505A62BBC97604]-worker) [n:127.0.0.1:37438_zb%2Fq c:collection1 s:shard2 r:core_node5 x:collection1] o.a.h.m.i.MetricsSystemImpl JobHistoryServer metrics system stopped.
   [junit4]   2> 79552 INFO  (SUITE-MorphlineGoLiveMiniMRTest-seed#[A8505A62BBC97604]-worker) [n:127.0.0.1:37438_zb%2Fq c:collection1 s:shard2 r:core_node5 x:collection1] o.a.h.m.i.MetricsSystemImpl JobHistoryServer metrics system shutdown complete.
   [junit4]   2> 79553 INFO  (SUITE-MorphlineGoLiveMiniMRTest-seed#[A8505A62BBC97604]-worker) [n:127.0.0.1:37438_zb%2Fq c:collection1 s:shard2 r:core_node5 x:collection1] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J1/temp/solr.hadoop.MorphlineGoLiveMiniMRTest_A8505A62BBC97604-001
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene54): {}, docValues:{}, sim=RandomSimilarityProvider(queryNorm=true,coord=crazy): {}, locale=fr_CA, timezone=Antarctica/Casey
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_80 (64-bit)/cpus=4,threads=9,free=97731144,total=343408640
   [junit4]   2> NOTE: All tests run in this JVM: [MorphlineReducerTest, MorphlineGoLiveMiniMRTest]
   [junit4] Completed [7/7] on J1 in 80.43s, 2 tests, 1 error <<< FAILURES!

[...truncated 13 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:253: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:467: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 7 suites, 35 tests, 1 suite-level error, 2 errors, 1 ignored [seed: A8505A62BBC97604]

Total time: 190 minutes 26 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1027 - Still Failing

[Apache Jenkins Server <je...@builds.apache.org>]

Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1027/

1 tests failed.
FAILED:  org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=11073, name=Thread-6280, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=11073, name=Thread-6280, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]
Caused by: java.lang.RuntimeException: org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:43455/_pxg/h/collection1
	at __randomizedtesting.SeedInfo.seed([59DA3974711B98F8]:0)
	at org.apache.solr.cloud.FullSolrCloudDistribCmdsTest$1IndexThread.run(FullSolrCloudDistribCmdsTest.java:645)
Caused by: org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:43455/_pxg/h/collection1
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:584)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:240)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:229)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:150)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:167)
	at org.apache.solr.cloud.FullSolrCloudDistribCmdsTest$1IndexThread.run(FullSolrCloudDistribCmdsTest.java:643)
Caused by: java.net.SocketTimeoutException: Read timed out
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.read(SocketInputStream.java:152)
	at java.net.SocketInputStream.read(SocketInputStream.java:122)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:160)
	at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:84)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:273)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
	at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
	at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283)
	at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251)
	at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197)
	at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)
	at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)
	at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685)
	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487)
	at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:479)
	... 5 more




Build Log:
[...truncated 10686 lines...]
   [junit4] Suite: org.apache.solr.cloud.FullSolrCloudDistribCmdsTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_59DA3974711B98F8-001/init-core-data-001
   [junit4]   2> 1356205 INFO  (SUITE-FullSolrCloudDistribCmdsTest-seed#[59DA3974711B98F8]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /_pxg/h
   [junit4]   2> 1356211 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 1356211 INFO  (Thread-6112) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1356211 INFO  (Thread-6112) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 1356311 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.ZkTestServer start zk server on port:54391
   [junit4]   2> 1356312 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1356312 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1356317 INFO  (zkCallback-957-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@766f28c0 name:ZooKeeperConnection Watcher:127.0.0.1:54391 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1356317 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1356317 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1356317 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 1356320 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1356320 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1356321 INFO  (zkCallback-958-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5b119dc1 name:ZooKeeperConnection Watcher:127.0.0.1:54391/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1356321 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1356322 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1356322 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 1356323 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 1356325 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 1356326 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 1356327 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 1356327 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 1356329 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
   [junit4]   2> 1356329 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 1356331 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1356331 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1356333 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 1356333 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 1356334 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 1356334 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 1356336 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 1356336 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 1356337 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 1356337 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 1356339 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1356339 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1356340 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1356341 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1356342 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 1356342 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 1356344 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 1356344 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 1356603 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_59DA3974711B98F8-001/control-001/cores/collection1
   [junit4]   2> 1356604 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 1356606 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@af97390{/_pxg/h,null,AVAILABLE}
   [junit4]   2> 1356608 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.e.j.s.ServerConnector Started ServerConnector@48b167be{HTTP/1.1}{127.0.0.1:43455}
   [junit4]   2> 1356608 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.e.j.s.Server Started @1359146ms
   [junit4]   2> 1356608 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/_pxg/h, hostPort=43455, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_59DA3974711B98F8-001/tempDir-001/control/data, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_59DA3974711B98F8-001/control-001/cores}
   [junit4]   2> 1356608 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@7b3cb2c6
   [junit4]   2> 1356608 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_59DA3974711B98F8-001/control-001'
   [junit4]   2> 1356609 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1356609 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1356609 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1356610 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1356610 INFO  (zkCallback-959-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@530350ee name:ZooKeeperConnection Watcher:127.0.0.1:54391/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1356611 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1356611 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1356612 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1356612 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_59DA3974711B98F8-001/control-001/solr.xml
   [junit4]   2> 1356624 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_59DA3974711B98F8-001/control-001/cores
   [junit4]   2> 1356624 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.CoreContainer New CoreContainer 293072664
   [junit4]   2> 1356624 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_59DA3974711B98F8-001/control-001]
   [junit4]   2> 1356624 WARN  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_59DA3974711B98F8-001/control-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_59DA3974711B98F8-001/control-001/lib
   [junit4]   2> 1356625 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1356626 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1356626 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1356626 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1356626 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:54391/solr
   [junit4]   2> 1356626 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 1356626 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1356627 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1356628 INFO  (zkCallback-961-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@641a406e name:ZooKeeperConnection Watcher:127.0.0.1:54391 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1356629 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1356630 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1356631 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [n:127.0.0.1:43455__pxg%2Fh    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1356632 INFO  (zkCallback-962-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@71722a5e name:ZooKeeperConnection Watcher:127.0.0.1:54391/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1356632 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [n:127.0.0.1:43455__pxg%2Fh    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1356633 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [n:127.0.0.1:43455__pxg%2Fh    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 1356635 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [n:127.0.0.1:43455__pxg%2Fh    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 1356637 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [n:127.0.0.1:43455__pxg%2Fh    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 1356639 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [n:127.0.0.1:43455__pxg%2Fh    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 1356641 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [n:127.0.0.1:43455__pxg%2Fh    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 1356642 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [n:127.0.0.1:43455__pxg%2Fh    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 1356644 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [n:127.0.0.1:43455__pxg%2Fh    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 1356645 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [n:127.0.0.1:43455__pxg%2Fh    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 1356646 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [n:127.0.0.1:43455__pxg%2Fh    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 1356647 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [n:127.0.0.1:43455__pxg%2Fh    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:43455__pxg%2Fh
   [junit4]   2> 1356647 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [n:127.0.0.1:43455__pxg%2Fh    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:43455__pxg%2Fh
   [junit4]   2> 1356648 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [n:127.0.0.1:43455__pxg%2Fh    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 1356649 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [n:127.0.0.1:43455__pxg%2Fh    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 1356650 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [n:127.0.0.1:43455__pxg%2Fh    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1356651 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [n:127.0.0.1:43455__pxg%2Fh    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94922873051480068-127.0.0.1:43455__pxg%2Fh-n_0000000000
   [junit4]   2> 1356651 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [n:127.0.0.1:43455__pxg%2Fh    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:43455__pxg%2Fh
   [junit4]   2> 1356651 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [n:127.0.0.1:43455__pxg%2Fh    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 1356652 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [n:127.0.0.1:43455__pxg%2Fh    ] o.a.s.c.Overseer Overseer (id=94922873051480068-127.0.0.1:43455__pxg%2Fh-n_0000000000) starting
   [junit4]   2> 1356654 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [n:127.0.0.1:43455__pxg%2Fh    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 1356659 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [n:127.0.0.1:43455__pxg%2Fh    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 1356659 INFO  (OverseerCollectionConfigSetProcessor-94922873051480068-127.0.0.1:43455__pxg%2Fh-n_0000000000) [n:127.0.0.1:43455__pxg%2Fh    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 1356659 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [n:127.0.0.1:43455__pxg%2Fh    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1356659 INFO  (OverseerStateUpdate-94922873051480068-127.0.0.1:43455__pxg%2Fh-n_0000000000) [n:127.0.0.1:43455__pxg%2Fh    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 1356676 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [n:127.0.0.1:43455__pxg%2Fh    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1356676 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [n:127.0.0.1:43455__pxg%2Fh    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1356677 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [n:127.0.0.1:43455__pxg%2Fh    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_59DA3974711B98F8-001/control-001/cores
   [junit4]   2> 1356678 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [n:127.0.0.1:43455__pxg%2Fh    ] o.a.s.c.SolrCore Created CoreDescriptor: {instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_59DA3974711B98F8-001/control-001/cores/collection1, dataDir=data/, schema=schema.xml, configSetProperties=configsetprops.json, collection=control_collection, config=solrconfig.xml, coreNodeName=, name=collection1, loadOnStartup=true, shard=, transient=false, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_59DA3974711B98F8-001/control-001/cores/collection1/}
   [junit4]   2> 1356678 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [n:127.0.0.1:43455__pxg%2Fh    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_59DA3974711B98F8-001/control-001/cores/collection1/
   [junit4]   2> 1356678 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [n:127.0.0.1:43455__pxg%2Fh    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 1356679 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [n:127.0.0.1:43455__pxg%2Fh    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 1356679 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [n:127.0.0.1:43455__pxg%2Fh    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1356679 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 1356679 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1356681 INFO  (OverseerStateUpdate-94922873051480068-127.0.0.1:43455__pxg%2Fh-n_0000000000) [n:127.0.0.1:43455__pxg%2Fh    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "state":"down",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "node_name":"127.0.0.1:43455__pxg%2Fh",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "base_url":"http://127.0.0.1:43455/_pxg/h",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "roles":null,
   [junit4]   2>   "core":"collection1"} current state version: 0
   [junit4]   2> 1356681 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 1356681 INFO  (OverseerStateUpdate-94922873051480068-127.0.0.1:43455__pxg%2Fh-n_0000000000) [n:127.0.0.1:43455__pxg%2Fh    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "state":"down",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "node_name":"127.0.0.1:43455__pxg%2Fh",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "base_url":"http://127.0.0.1:43455/_pxg/h",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "roles":null,
   [junit4]   2>   "core":"collection1"}
   [junit4]   2> 1356682 INFO  (OverseerStateUpdate-94922873051480068-127.0.0.1:43455__pxg%2Fh-n_0000000000) [n:127.0.0.1:43455__pxg%2Fh    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 1356682 INFO  (OverseerStateUpdate-94922873051480068-127.0.0.1:43455__pxg%2Fh-n_0000000000) [n:127.0.0.1:43455__pxg%2Fh    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 1356683 INFO  (zkCallback-962-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1357682 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1357682 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 1357682 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 1357682 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 1357683 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1357683 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_59DA3974711B98F8-001/control-001/cores/collection1'
   [junit4]   2> 1357683 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1357683 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1357685 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 1357693 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 1357705 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 1357749 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1357754 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1357946 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 1357948 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 1357949 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1357951 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1357967 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 1357967 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 1357967 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 1357968 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_59DA3974711B98F8-001/control-001/cores/collection1], dataDir=[null]
   [junit4]   2> 1357968 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@288a2593
   [junit4]   2> 1357968 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_59DA3974711B98F8-001/control-001/cores/collection1/data
   [junit4]   2> 1357968 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_59DA3974711B98F8-001/control-001/cores/collection1/data/index/
   [junit4]   2> 1357968 WARN  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_59DA3974711B98F8-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 1357968 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_59DA3974711B98F8-001/control-001/cores/collection1/data/index
   [junit4]   2> 1357969 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=33, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.6863981439809271]
   [junit4]   2> 1357969 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@75648327 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@19deb1a7),segFN=segments_1,generation=1}
   [junit4]   2> 1357969 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 1357970 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_59DA3974711B98F8-001/control-001/cores/collection1/data/
   [junit4]   2> 1357970 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_59DA3974711B98F8-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 1357973 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 1357973 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 1357973 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 1357973 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1357973 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1357973 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 1357974 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1357974 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1357974 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 1357974 WARN  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
   [junit4]   2> 1357977 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1357978 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1357979 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1357980 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1357983 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: standard,/admin/luke,/admin/logging,/admin/mbeans,/admin/threads,/replication,/dump,/update/csv,/get,/admin/file,/admin/plugins,/admin/properties,/admin/system,/schema,/config,/update,/admin/segments,/update/json,/admin/ping,/update/json/docs
   [junit4]   2> 1357984 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 1357984 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1357984 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1357985 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 1357985 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 1357985 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@2ddbeec0
   [junit4]   2> 1357986 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@75648327 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@19deb1a7),segFN=segments_1,generation=1}
   [junit4]   2> 1357986 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 1357986 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@380eb10a[collection1] main
   [junit4]   2> 1357986 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 1357992 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1357992 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 1357993 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 1357993 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 1357993 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1357993 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 1357994 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1357994 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 1357994 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1357994 INFO  (searcherExecutor-4047-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@380eb10a[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1357994 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 1357994 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 1357994 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 1357994 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1518765970691194880
   [junit4]   2> 1357996 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 1.0ms to seed version buckets with highest version 1518765970691194880
   [junit4]   2> 1357996 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 1357997 INFO  (coreLoadExecutor-4046-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 1357997 INFO  (coreZkRegister-4040-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:43455/_pxg/h collection:control_collection shard:shard1
   [junit4]   2> 1357998 INFO  (coreZkRegister-4040-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 1358000 INFO  (coreZkRegister-4040-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94922873051480068-core_node1-n_0000000000
   [junit4]   2> 1358001 INFO  (coreZkRegister-4040-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 1358002 INFO  (coreZkRegister-4040-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1358002 INFO  (coreZkRegister-4040-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1358002 INFO  (coreZkRegister-4040-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:43455/_pxg/h/collection1/
   [junit4]   2> 1358002 INFO  (coreZkRegister-4040-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 1358002 INFO  (OverseerStateUpdate-94922873051480068-127.0.0.1:43455__pxg%2Fh-n_0000000000) [n:127.0.0.1:43455__pxg%2Fh    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 1358002 INFO  (coreZkRegister-4040-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:43455/_pxg/h/collection1/ has no replicas
   [junit4]   2> 1358003 INFO  (coreZkRegister-4040-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 1358005 INFO  (coreZkRegister-4040-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 1358006 INFO  (coreZkRegister-4040-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:43455/_pxg/h/collection1/ shard1
   [junit4]   2> 1358007 INFO  (OverseerStateUpdate-94922873051480068-127.0.0.1:43455__pxg%2Fh-n_0000000000) [n:127.0.0.1:43455__pxg%2Fh    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:43455/_pxg/h",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 1
   [junit4]   2> 1358109 INFO  (zkCallback-962-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1358157 INFO  (coreZkRegister-4040-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:43455/_pxg/h/collection1/ and leader is http://127.0.0.1:43455/_pxg/h/collection1/
   [junit4]   2> 1358157 INFO  (coreZkRegister-4040-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:43455/_pxg/h
   [junit4]   2> 1358157 INFO  (coreZkRegister-4040-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 1358158 INFO  (coreZkRegister-4040-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 1358158 INFO  (coreZkRegister-4040-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1) [n:127.0.0.1:43455__pxg%2Fh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1358159 INFO  (OverseerStateUpdate-94922873051480068-127.0.0.1:43455__pxg%2Fh-n_0000000000) [n:127.0.0.1:43455__pxg%2Fh    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "state":"active",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "node_name":"127.0.0.1:43455__pxg%2Fh",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "base_url":"http://127.0.0.1:43455/_pxg/h",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1"} current state version: 2
   [junit4]   2> 1358160 INFO  (OverseerStateUpdate-94922873051480068-127.0.0.1:43455__pxg%2Fh-n_0000000000) [n:127.0.0.1:43455__pxg%2Fh    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "state":"active",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "node_name":"127.0.0.1:43455__pxg%2Fh",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "base_url":"http://127.0.0.1:43455/_pxg/h",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1"}
   [junit4]   2> 1358181 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1358181 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1358183 INFO  (zkCallback-964-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@56e19432 name:ZooKeeperConnection Watcher:127.0.0.1:54391/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1358183 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1358184 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1358184 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1358185 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 1358262 INFO  (zkCallback-964-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1358262 INFO  (zkCallback-962-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1358441 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_59DA3974711B98F8-001/shard-1-001/cores/collection1
   [junit4]   2> 1358441 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_59DA3974711B98F8-001/shard-1-001
   [junit4]   2> 1358444 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 1358446 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@6cd2d6b4{/_pxg/h,null,AVAILABLE}
   [junit4]   2> 1358446 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.e.j.s.ServerConnector Started ServerConnector@7fe964e5{HTTP/1.1}{127.0.0.1:44618}
   [junit4]   2> 1358446 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.e.j.s.Server Started @1360984ms
   [junit4]   2> 1358447 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_59DA3974711B98F8-001/shard-1-001/cores, hostContext=/_pxg/h, solrconfig=solrconfig.xml, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_59DA3974711B98F8-001/tempDir-001/jetty1, hostPort=44618}
   [junit4]   2> 1358447 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@7b3cb2c6
   [junit4]   2> 1358447 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_59DA3974711B98F8-001/shard-1-001'
   [junit4]   2> 1358447 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1358447 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1358447 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1358448 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1358449 INFO  (zkCallback-965-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@3e630cf9 name:ZooKeeperConnection Watcher:127.0.0.1:54391/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1358449 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1358449 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1358450 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1358450 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_59DA3974711B98F8-001/shard-1-001/solr.xml
   [junit4]   2> 1358462 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_59DA3974711B98F8-001/shard-1-001/cores
   [junit4]   2> 1358462 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.CoreContainer New CoreContainer 150031034
   [junit4]   2> 1358463 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_59DA3974711B98F8-001/shard-1-001]
   [junit4]   2> 1358463 WARN  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_59DA3974711B98F8-001/shard-1-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_59DA3974711B98F8-001/shard-1-001/lib
   [junit4]   2> 1358463 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1358464 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1358464 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1358464 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1358464 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:54391/solr
   [junit4]   2> 1358465 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 1358465 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1358465 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1358466 INFO  (zkCallback-967-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5ba85301 name:ZooKeeperConnection Watcher:127.0.0.1:54391 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1358467 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1358467 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1358469 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [n:127.0.0.1:44618__pxg%2Fh    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1358470 INFO  (zkCallback-968-thread-1-processing-n:127.0.0.1:44618__pxg%2Fh) [n:127.0.0.1:44618__pxg%2Fh    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@753257f3 name:ZooKeeperConnection Watcher:127.0.0.1:54391/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1358470 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [n:127.0.0.1:44618__pxg%2Fh    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1358475 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [n:127.0.0.1:44618__pxg%2Fh    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1359480 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [n:127.0.0.1:44618__pxg%2Fh    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:44618__pxg%2Fh
   [junit4]   2> 1359480 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [n:127.0.0.1:44618__pxg%2Fh    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:44618__pxg%2Fh
   [junit4]   2> 1359481 INFO  (zkCallback-968-thread-1-processing-n:127.0.0.1:44618__pxg%2Fh) [n:127.0.0.1:44618__pxg%2Fh    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1359481 INFO  (zkCallback-964-thread-1) [    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1359481 INFO  (zkCallback-962-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1359483 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [n:127.0.0.1:44618__pxg%2Fh    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1359484 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [n:127.0.0.1:44618__pxg%2Fh    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94922873051480072-127.0.0.1:44618__pxg%2Fh-n_0000000001
   [junit4]   2> 1359484 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [n:127.0.0.1:44618__pxg%2Fh    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/94922873051480068-127.0.0.1:43455__pxg%2Fh-n_0000000000 to know if I could be the leader
   [junit4]   2> 1359496 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [n:127.0.0.1:44618__pxg%2Fh    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1359496 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [n:127.0.0.1:44618__pxg%2Fh    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1359497 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [n:127.0.0.1:44618__pxg%2Fh    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_59DA3974711B98F8-001/shard-1-001/cores
   [junit4]   2> 1359498 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [n:127.0.0.1:44618__pxg%2Fh    ] o.a.s.c.SolrCore Created CoreDescriptor: {loadOnStartup=true, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_59DA3974711B98F8-001/shard-1-001/cores/collection1, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_59DA3974711B98F8-001/shard-1-001/cores/collection1/, schema=schema.xml, collection=collection1, config=solrconfig.xml, dataDir=data/, shard=, coreNodeName=, configSetProperties=configsetprops.json, transient=false, name=collection1}
   [junit4]   2> 1359498 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [n:127.0.0.1:44618__pxg%2Fh    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_59DA3974711B98F8-001/shard-1-001/cores/collection1/
   [junit4]   2> 1359498 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [n:127.0.0.1:44618__pxg%2Fh    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 1359499 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [n:127.0.0.1:44618__pxg%2Fh    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 1359499 INFO  (TEST-FullSolrCloudDistribCmdsTest.test-seed#[59DA3974711B98F8]) [n:127.0.0.1:44618__pxg%2Fh    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1359499 INFO  (coreLoadExecutor-4057-thread-1-processing-n:127.0.0.1:44618__pxg%2Fh) [n:127.0.0.1:44618__pxg%2Fh c:collection1   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 1359499 INFO  (coreLoadExecutor-4057-thread-1-processing-n:127.0.0.1:44618__pxg%2Fh) [n:127.0.0.1:44618__pxg%2Fh c:collection1   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1359501 INFO  (coreLoadExecutor-4057-thread-1-processing-n:127.0.0.1:44618__pxg%2Fh) [n:127.0.0.1:44618__pxg%2Fh c:collection1   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 1359501 INFO  (OverseerStateUpdate-94922873051480068-127.0.0.1:43455__pxg%2Fh-n_0000000000) [n:127.0.0.1:43455__pxg%2Fh    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "state":"down",
   [junit4]   2>   "numShards":"3",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "node_name":"127.0.0.1:44618__pxg%2Fh",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "base_url":"http://127.0.0.1:44618/_pxg/h",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "roles":null,
   [junit4]   2>   "core":"collection1"} current state version: 3
   [junit4]   2> 1359501 INFO  (OverseerStateUpdate-94922873051480068-127.0.0.1:43455__pxg%2Fh-n_0000000000) [n:127.0.0.1:43455__pxg%2Fh    ] o.a.s.c.o.ReplicaMutator Update state numShards=3 message={
   [junit4]   2>   "state":"down",
   [junit4]   2>   "numShards":"3",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "node_name":"127.0.0.1:44618__pxg%2Fh",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "base_url":"http://127.0.0.1:44618/_pxg/h",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "roles":null,
   [junit4]   2>   "core":"collection1"}
   [junit4]   2> 1359501 INFO  (OverseerStateUpdate-94922873051480068-127.0.0.1:43455__pxg%2Fh-n_0000000000) [n:127.0.0.1:43455__pxg%2Fh    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 1359502 INFO  (OverseerStateUpdate-94922873051480068-127.0.0.1:43455__pxg%2Fh-n_0000000000) [n:127.0.0.1:43455__pxg%2Fh    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard3
   [junit4]   2> 1359604 INFO  (zkCallback-962-thread-1-processing-n:127.0.0.1:43455__pxg%2Fh) [n:127.0.0.1:43455__pxg%2Fh    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1359604 INFO  (zkCallback-964-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1359604 INFO  (zkCallback-968-thread-1-processing-n:127.0.0.1:44618__pxg%2Fh) [n:127.0.0.1:44618__pxg%2Fh    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1360501 INFO  (coreLoadExecutor-4057-thread-1-processing-n:127.0.0.1:44618__pxg%2Fh) [n:127.0.0.1:44618__pxg%2Fh c:collection1   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1360501 INFO  (coreLoadExecutor-4057-thread-1-processing-n:127.0.0.1:44618__pxg%2Fh) [n:127.0.0.1:44618__pxg%2Fh c:collection1   x:collection1] o.a.s.c.ZkController Check for collection zkNode:collection1
   [junit4]   2> 1360502 INFO  (coreLoadExecutor-4057-thread-1-processing-n:127.0.0.1:44618__pxg%2Fh) [n:127.0.0.1:44618__pxg%2Fh c:collection1   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 1360502 INFO  (coreLoadExecutor-4057-thread-1-processing-n:127.0.0.1:44618__pxg%2Fh) [n:127.0.0.1:44618__pxg%2Fh c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/collection1
   [junit4]   2> 1360503 INFO  (coreLoadExecutor-4057-thread-1-processing-n:127.0.0.1:44618__pxg%2Fh) [n:127.0.0.1:44618__pxg%2Fh c:collection1   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1360503 INFO  (coreLoadExecutor-4057-thread-1-processing-n:127.0.0.1:44618__pxg%2Fh) [n:127.0.0.1:44618__pxg%2Fh c:collection1   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_59DA3974711B98F8-001/shard-1-001/cores/collection1'
   [junit4]   2> 1360503 INFO  (coreLoadExecutor-4057-thread-1-processing-n:127.0.0.1:44618__pxg%2Fh) [n:127.0.0.1:44618__pxg%2Fh c:collection1   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1360503 INFO  (coreLoadExecutor-4057-thread-1-processing-n:127.0.0.1:44618__pxg%2Fh) [n:127.0.0.1:44618__pxg%2Fh c:collection1   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1360505 INFO  (coreLoadExecutor-4057-thread-1-processing-n:127.0.0.1:44618__pxg%2Fh) [n:127.0.0.1:44618__pxg%2Fh c:collection1   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 1360511 INFO  (coreLoadExecutor-4057-thread-1-processing-n:127.0.0.1:44618__pxg%2Fh) [n:127.0.0.1:44618__pxg%2Fh c:collection1   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 1360527 INFO  (coreLoadExecutor-4057-thread-1-processing-n:127.0.0.1:44618__pxg%2Fh) [n:127.0.0.1:44618__pxg%2Fh c:collection1   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 1360553 INFO  (coreLoadExecutor-4057-thread-1-processing-n:127.0.0.1:44618__pxg%2Fh) [n:127.0.0.1:44618__pxg%2Fh c:collection1   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1360559 INFO  (coreLoadExecutor-4057-thread-1-processing-n:127.0.0.1:44618__pxg%2Fh) [n:127.0.0.1:44618__pxg%2Fh c:collection1   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1360762 INFO  (coreLoadExecutor-4057-thread-1-processing-n:127.0.0.1:44618__pxg%2Fh) [n:127.0.0.1:44618__pxg%2Fh c:collection1   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 1360764 INFO  (coreLoadExecutor-4057-thread-1-processing-n:127.0.0.1:44618__pxg%2Fh) [n:127.0.0.1:44618__pxg%2Fh c:collection1   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 1360765 INFO  (coreLoadExecutor-4057-thread-1-processing-n:127.0.0.1:44618__pxg%2Fh) [n:127.0.0.1:44618__pxg%2Fh c:collection1   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1360768 INFO  (coreLoadExecutor-4057-thread-1-processing-n:127.0.0.1:44618__pxg%2Fh) [n:127.0.0.1:44618__pxg%2Fh c:collection1   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1360783 INFO  (coreLoadExecutor-4057-thread-1-processing-n:127.0.0.1:44618__pxg%2Fh) [n:127.0.0.1:44618__pxg%2Fh c:collection1   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 1360783 INFO  (coreLoadExecutor-4057-thread-1-processing-n:127.0.0.1:44618__pxg%2Fh) [n:127.0.0.1:44618__pxg%2Fh c:collection1   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection collection1
   [junit4]   2> 1360783 INFO  (coreLoadExecutor-4057-thread-1-processing-n:127.0.0.1:44618__pxg%2Fh) [n:127.0.0.1:44618__pxg%2Fh c:collection1 s:shard3 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 1360783 INFO  (coreLoadExecutor-4057-thread-1-processing-n:127.0.0.1:44618__pxg%2Fh) [n:127.0.0.1:44618__pxg%2Fh c:collection1 s:shard3 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_59DA3974711B98F8-001/shard-1-001/cores/collection1], dataDir=[null]
   [junit4]   2> 1360783 INFO  (coreLoadExecutor-4057-thread-1-processing-n:127.0.0.1:44618__pxg%2Fh) [n:127.0.0.1:44618__pxg%2Fh c:collection1 s:shard3 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@288a2593
   [junit4]   2> 1360784 INFO  (coreLoadExecutor-4057-thread-1-processing-n:127.0.0.1:44618__pxg%2Fh) [n:127.0.0.1:44618__pxg%2Fh c:collection1 s:shard3 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_59DA3974711B98F8-001/shard-1-001/cores/collection1/data
   [junit4]   2> 1360784 INFO  (coreLoadExecutor-4057-thread-1-processing-n:127.0.0.1:44618__pxg%2Fh) [n:127.0.0.1:44618__pxg%2Fh c:collection1 s:shard3 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_59DA3974711B98F8-001/shard-1-001/cores/collection1/data/index/
   [junit4]   2> 1360784 WARN  (coreLoadExecutor-4057-thread-1-processing-n:127.0.0.1:44618__pxg%2Fh) [n:127.0.0.1:44618__pxg%2Fh c:collection1 s:shard3 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_59DA3974711B98F8-001/shard-1-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 1360784 INFO  (coreLoadExecutor-4057-thread-1-processing-n:127.0.0.1:44618__pxg%2Fh) [n:127.0.0.1:44618__pxg%2Fh c:collection1 s:shard3 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_59DA3974711B98F8-001/shard-1-001/cores/collection1/data/index
   [junit4]   2> 1360784 INFO  (coreLoadExecutor-4057-thread-1-processing-n:127.0.0.1:44618__pxg%2Fh) [n:127.0.0.1:44618__pxg%2Fh c:collection1 s:shard3 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=33, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.6863981439809271]
   [junit4]   2> 1360785 INFO  (coreLoadExecutor-4057-thread-1-processing-n:127.0.0

[...truncated too long message...]

.process(SolrRequest.java:167)
   [junit4]    > 	at org.apache.solr.cloud.FullSolrCloudDistribCmdsTest$1IndexThread.run(FullSolrCloudDistribCmdsTest.java:643)
   [junit4]    > Caused by: java.net.SocketTimeoutException: Read timed out
   [junit4]    > 	at java.net.SocketInputStream.socketRead0(Native Method)
   [junit4]    > 	at java.net.SocketInputStream.read(SocketInputStream.java:152)
   [junit4]    > 	at java.net.SocketInputStream.read(SocketInputStream.java:122)
   [junit4]    > 	at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:160)
   [junit4]    > 	at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:84)
   [junit4]    > 	at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:273)
   [junit4]    > 	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
   [junit4]    > 	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
   [junit4]    > 	at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
   [junit4]    > 	at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283)
   [junit4]    > 	at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251)
   [junit4]    > 	at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197)
   [junit4]    > 	at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)
   [junit4]    > 	at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)
   [junit4]    > 	at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685)
   [junit4]    > 	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487)
   [junit4]    > 	at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
   [junit4]    > 	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
   [junit4]    > 	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
   [junit4]    > 	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:479)
   [junit4]    > 	... 5 moreThrowable #3: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=11075, name=Thread-6282, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]
   [junit4]    > Caused by: java.lang.RuntimeException: org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:43455/_pxg/h/collection1
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([59DA3974711B98F8]:0)
   [junit4]    > 	at org.apache.solr.cloud.FullSolrCloudDistribCmdsTest$1IndexThread.run(FullSolrCloudDistribCmdsTest.java:645)
   [junit4]    > Caused by: org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:43455/_pxg/h/collection1
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:584)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:240)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:229)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:150)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:167)
   [junit4]    > 	at org.apache.solr.cloud.FullSolrCloudDistribCmdsTest$1IndexThread.run(FullSolrCloudDistribCmdsTest.java:643)
   [junit4]    > Caused by: java.net.SocketTimeoutException: Read timed out
   [junit4]    > 	at java.net.SocketInputStream.socketRead0(Native Method)
   [junit4]    > 	at java.net.SocketInputStream.read(SocketInputStream.java:152)
   [junit4]    > 	at java.net.SocketInputStream.read(SocketInputStream.java:122)
   [junit4]    > 	at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:160)
   [junit4]    > 	at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:84)
   [junit4]    > 	at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:273)
   [junit4]    > 	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
   [junit4]    > 	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
   [junit4]    > 	at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
   [junit4]    > 	at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283)
   [junit4]    > 	at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251)
   [junit4]    > 	at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197)
   [junit4]    > 	at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)
   [junit4]    > 	at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)
   [junit4]    > 	at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685)
   [junit4]    > 	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487)
   [junit4]    > 	at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
   [junit4]    > 	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
   [junit4]    > 	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
   [junit4]    > 	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:479)
   [junit4]    > 	... 5 more
   [junit4]   2> 1741997 INFO  (SUITE-FullSolrCloudDistribCmdsTest-seed#[59DA3974711B98F8]-worker) [n:127.0.0.1:57558__pxg%2Fh c:collection2 s:shard3 r:core_node17 x:collection2_shard3_replica2] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: test params are: codec=CheapBastard, sim=RandomSimilarityProvider(queryNorm=true,coord=no): {}, locale=es_PY, timezone=America/Maceio
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_80 (64-bit)/cpus=4,threads=1,free=266325784,total=477102080
   [junit4]   2> NOTE: All tests run in this JVM: [TestIndexingPerformance, RequestHandlersTest, TestLeaderElectionZkExpiry, DateRangeFieldTest, FieldMutatingUpdateProcessorTest, TestReRankQParserPlugin, TestCollationField, TestGroupingSearch, TestPHPSerializedResponseWriter, AssignTest, TestSuggestSpellingConverter, TestSweetSpotSimilarityFactory, TestFieldTypeCollectionResource, AtomicUpdatesTest, TestRandomCollapseQParserPlugin, TestFileDictionaryLookup, TestSolrConfigHandlerConcurrent, ForceLeaderTest, LeaderInitiatedRecoveryOnShardRestartTest, ScriptEngineTest, CloneFieldUpdateProcessorFactoryTest, BadIndexSchemaTest, TestJmxMonitoredMap, CoreAdminHandlerTest, HdfsThreadLeakTest, TestChildDocTransformer, InfoHandlerTest, TestLRUStatsCache, TestExactSharedStatsCache, OverseerTaskQueueTest, RecoveryZkTest, SaslZkACLProviderTest, TestSearchPerf, CircularListTest, AliasIntegrationTest, PluginInfoTest, StatelessScriptUpdateProcessorFactoryTest, TestBulkSchemaConcurrent, FieldAnalysisRequestHandlerTest, TestPerFieldSimilarity, MergeStrategyTest, TestSolrIndexConfig, BigEndianAscendingWordSerializerTest, PrimitiveFieldTypeTest, ResponseHeaderTest, TestReplicaProperties, TestRecovery, ShowFileRequestHandlerTest, TestFastWriter, TestMiniSolrCloudClusterBase, TestTrackingShardHandlerFactory, TestStressReorder, SharedFSAutoReplicaFailoverTest, DistribCursorPagingTest, RecoveryAfterSoftCommitTest, TestCharFilters, TestSolrQueryParser, TestReversedWildcardFilterFactory, OverseerStatusTest, HdfsCollectionsAPIDistributedZkTest, AutoCommitTest, CurrencyFieldXmlFileTest, IgnoreCommitOptimizeUpdateProcessorFactoryTest, DistributedQueryComponentCustomSortTest, FullSolrCloudDistribCmdsTest]
   [junit4] Completed [239/551] on J2 in 386.62s, 1 test, 1 error <<< FAILURES!

[...truncated 1051 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 551 suites (7 ignored), 2177 tests, 1 error, 32 ignored (18 assumptions) [seed: 59DA3974711B98F8]

Total time: 210 minutes 46 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1026 - Still Failing

[Apache Jenkins Server <je...@builds.apache.org>]

Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1026/

1 tests failed.
FAILED:  org.apache.lucene.index.TestDuelingCodecsAtNight.testBigEquals

Error Message:
this IndexWriter is closed

Stack Trace:
org.apache.lucene.store.AlreadyClosedException: this IndexWriter is closed
	at org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:720)
	at org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:734)
	at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1473)
	at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1256)
	at org.apache.lucene.index.RandomIndexWriter.addDocument(RandomIndexWriter.java:170)
	at org.apache.lucene.index.TestDuelingCodecs.createRandomIndex(TestDuelingCodecs.java:139)
	at org.apache.lucene.index.TestDuelingCodecsAtNight.testBigEquals(TestDuelingCodecsAtNight.java:34)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.OutOfMemoryError: Java heap space
	at org.apache.lucene.util.fst.BytesStore.skipBytes(BytesStore.java:309)
	at org.apache.lucene.util.fst.FST.addNode(FST.java:792)
	at org.apache.lucene.util.fst.NodeHash.add(NodeHash.java:126)
	at org.apache.lucene.util.fst.Builder.compileNode(Builder.java:215)
	at org.apache.lucene.util.fst.Builder.finish(Builder.java:500)
	at org.apache.lucene.codecs.memory.MemoryPostingsFormat$TermsWriter.finish(MemoryPostingsFormat.java:271)
	at org.apache.lucene.codecs.memory.MemoryPostingsFormat$MemoryFieldsConsumer.write(MemoryPostingsFormat.java:405)
	at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsWriter.write(PerFieldPostingsFormat.java:198)
	at org.apache.lucene.codecs.FieldsConsumer.merge(FieldsConsumer.java:105)
	at org.apache.lucene.index.SegmentMerger.mergeTerms(SegmentMerger.java:193)
	at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:95)
	at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4075)
	at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3655)
	at org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:588)
	at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:626)




Build Log:
[...truncated 1753 lines...]
   [junit4] Suite: org.apache.lucene.index.TestDuelingCodecsAtNight
   [junit4]   2> نوف 24, 2015 2:21:42 ص com.carrotsearch.randomizedtesting.RandomizedRunner$QueueUncaughtExceptionsHandler uncaughtException
   [junit4]   2> WARNING: Uncaught exception in thread: Thread[Lucene Merge Thread #14,5,TGRP-TestDuelingCodecsAtNight]
   [junit4]   2> org.apache.lucene.index.MergePolicy$MergeException: java.lang.OutOfMemoryError: Java heap space
   [junit4]   2> 	at __randomizedtesting.SeedInfo.seed([509A95B358E09389]:0)
   [junit4]   2> 	at org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:668)
   [junit4]   2> 	at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:648)
   [junit4]   2> Caused by: java.lang.OutOfMemoryError: Java heap space
   [junit4]   2> 	at org.apache.lucene.util.fst.BytesStore.skipBytes(BytesStore.java:309)
   [junit4]   2> 	at org.apache.lucene.util.fst.FST.pack(FST.java:1762)
   [junit4]   2> 	at org.apache.lucene.util.fst.Builder.finish(Builder.java:503)
   [junit4]   2> 	at org.apache.lucene.codecs.memory.MemoryPostingsFormat$TermsWriter.finish(MemoryPostingsFormat.java:271)
   [junit4]   2> 	at org.apache.lucene.codecs.memory.MemoryPostingsFormat$MemoryFieldsConsumer.write(MemoryPostingsFormat.java:405)
   [junit4]   2> 	at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsWriter.write(PerFieldPostingsFormat.java:198)
   [junit4]   2> 	at org.apache.lucene.codecs.FieldsConsumer.merge(FieldsConsumer.java:105)
   [junit4]   2> 	at org.apache.lucene.index.SegmentMerger.mergeTerms(SegmentMerger.java:193)
   [junit4]   2> 	at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:95)
   [junit4]   2> 	at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4075)
   [junit4]   2> 	at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3655)
   [junit4]   2> 	at org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:588)
   [junit4]   2> 	at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:626)
   [junit4]   2> 
   [junit4]   2> نوف 24, 2015 2:21:42 ص com.carrotsearch.randomizedtesting.RandomizedRunner$QueueUncaughtExceptionsHandler uncaughtException
   [junit4]   2> WARNING: Uncaught exception in thread: Thread[Lucene Merge Thread #15,5,TGRP-TestDuelingCodecsAtNight]
   [junit4]   2> org.apache.lucene.index.MergePolicy$MergeException: java.lang.OutOfMemoryError: Java heap space
   [junit4]   2> 	at __randomizedtesting.SeedInfo.seed([509A95B358E09389]:0)
   [junit4]   2> 	at org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:668)
   [junit4]   2> 	at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:648)
   [junit4]   2> Caused by: java.lang.OutOfMemoryError: Java heap space
   [junit4]   2> 	at org.apache.lucene.util.fst.BytesStore.skipBytes(BytesStore.java:309)
   [junit4]   2> 	at org.apache.lucene.util.fst.FST.addNode(FST.java:792)
   [junit4]   2> 	at org.apache.lucene.util.fst.NodeHash.add(NodeHash.java:126)
   [junit4]   2> 	at org.apache.lucene.util.fst.Builder.compileNode(Builder.java:215)
   [junit4]   2> 	at org.apache.lucene.util.fst.Builder.finish(Builder.java:500)
   [junit4]   2> 	at org.apache.lucene.codecs.memory.MemoryPostingsFormat$TermsWriter.finish(MemoryPostingsFormat.java:271)
   [junit4]   2> 	at org.apache.lucene.codecs.memory.MemoryPostingsFormat$MemoryFieldsConsumer.write(MemoryPostingsFormat.java:405)
   [junit4]   2> 	at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsWriter.write(PerFieldPostingsFormat.java:198)
   [junit4]   2> 	at org.apache.lucene.codecs.FieldsConsumer.merge(FieldsConsumer.java:105)
   [junit4]   2> 	at org.apache.lucene.index.SegmentMerger.mergeTerms(SegmentMerger.java:193)
   [junit4]   2> 	at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:95)
   [junit4]   2> 	at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4075)
   [junit4]   2> 	at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3655)
   [junit4]   2> 	at org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:588)
   [junit4]   2> 	at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:626)
   [junit4]   2> 
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestDuelingCodecsAtNight -Dtests.method=testBigEquals -Dtests.seed=509A95B358E09389 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=ar_EG -Dtests.timezone=Indian/Maldives -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR   1041s J0 | TestDuelingCodecsAtNight.testBigEquals <<<
   [junit4]    > Throwable #1: org.apache.lucene.store.AlreadyClosedException: this IndexWriter is closed
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:720)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:734)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1473)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1256)
   [junit4]    > 	at org.apache.lucene.index.RandomIndexWriter.addDocument(RandomIndexWriter.java:170)
   [junit4]    > 	at org.apache.lucene.index.TestDuelingCodecs.createRandomIndex(TestDuelingCodecs.java:139)
   [junit4]    > 	at org.apache.lucene.index.TestDuelingCodecsAtNight.testBigEquals(TestDuelingCodecsAtNight.java:34)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]    > Caused by: java.lang.OutOfMemoryError: Java heap space
   [junit4]    > 	at org.apache.lucene.util.fst.BytesStore.skipBytes(BytesStore.java:309)
   [junit4]    > 	at org.apache.lucene.util.fst.FST.addNode(FST.java:792)
   [junit4]    > 	at org.apache.lucene.util.fst.NodeHash.add(NodeHash.java:126)
   [junit4]    > 	at org.apache.lucene.util.fst.Builder.compileNode(Builder.java:215)
   [junit4]    > 	at org.apache.lucene.util.fst.Builder.finish(Builder.java:500)
   [junit4]    > 	at org.apache.lucene.codecs.memory.MemoryPostingsFormat$TermsWriter.finish(MemoryPostingsFormat.java:271)
   [junit4]    > 	at org.apache.lucene.codecs.memory.MemoryPostingsFormat$MemoryFieldsConsumer.write(MemoryPostingsFormat.java:405)
   [junit4]    > 	at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsWriter.write(PerFieldPostingsFormat.java:198)
   [junit4]    > 	at org.apache.lucene.codecs.FieldsConsumer.merge(FieldsConsumer.java:105)
   [junit4]    > 	at org.apache.lucene.index.SegmentMerger.mergeTerms(SegmentMerger.java:193)
   [junit4]    > 	at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:95)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4075)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3655)
   [junit4]    > 	at org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:588)
   [junit4]    > 	at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:626)Throwable #2: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=5879, name=Lucene Merge Thread #14, state=RUNNABLE, group=TGRP-TestDuelingCodecsAtNight]
   [junit4]    > Caused by: org.apache.lucene.index.MergePolicy$MergeException: java.lang.OutOfMemoryError: Java heap space
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([509A95B358E09389]:0)
   [junit4]    > 	at org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:668)
   [junit4]    > 	at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:648)
   [junit4]    > Caused by: java.lang.OutOfMemoryError: Java heap space
   [junit4]    > 	at org.apache.lucene.util.fst.BytesStore.skipBytes(BytesStore.java:309)
   [junit4]    > 	at org.apache.lucene.util.fst.FST.pack(FST.java:1762)
   [junit4]    > 	at org.apache.lucene.util.fst.Builder.finish(Builder.java:503)
   [junit4]    > 	at org.apache.lucene.codecs.memory.MemoryPostingsFormat$TermsWriter.finish(MemoryPostingsFormat.java:271)
   [junit4]    > 	at org.apache.lucene.codecs.memory.MemoryPostingsFormat$MemoryFieldsConsumer.write(MemoryPostingsFormat.java:405)
   [junit4]    > 	at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsWriter.write(PerFieldPostingsFormat.java:198)
   [junit4]    > 	at org.apache.lucene.codecs.FieldsConsumer.merge(FieldsConsumer.java:105)
   [junit4]    > 	at org.apache.lucene.index.SegmentMerger.mergeTerms(SegmentMerger.java:193)
   [junit4]    > 	at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:95)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4075)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3655)
   [junit4]    > 	at org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:588)
   [junit4]    > 	at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:626)Throwable #3: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=5880, name=Lucene Merge Thread #15, state=RUNNABLE, group=TGRP-TestDuelingCodecsAtNight]
   [junit4]    > Caused by: org.apache.lucene.index.MergePolicy$MergeException: java.lang.OutOfMemoryError: Java heap space
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([509A95B358E09389]:0)
   [junit4]    > 	at org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:668)
   [junit4]    > 	at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:648)
   [junit4]    > Caused by: java.lang.OutOfMemoryError: Java heap space
   [junit4]    > 	at org.apache.lucene.util.fst.BytesStore.skipBytes(BytesStore.java:309)
   [junit4]    > 	at org.apache.lucene.util.fst.FST.addNode(FST.java:792)
   [junit4]    > 	at org.apache.lucene.util.fst.NodeHash.add(NodeHash.java:126)
   [junit4]    > 	at org.apache.lucene.util.fst.Builder.compileNode(Builder.java:215)
   [junit4]    > 	at org.apache.lucene.util.fst.Builder.finish(Builder.java:500)
   [junit4]    > 	at org.apache.lucene.codecs.memory.MemoryPostingsFormat$TermsWriter.finish(MemoryPostingsFormat.java:271)
   [junit4]    > 	at org.apache.lucene.codecs.memory.MemoryPostingsFormat$MemoryFieldsConsumer.write(MemoryPostingsFormat.java:405)
   [junit4]    > 	at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsWriter.write(PerFieldPostingsFormat.java:198)
   [junit4]    > 	at org.apache.lucene.codecs.FieldsConsumer.merge(FieldsConsumer.java:105)
   [junit4]    > 	at org.apache.lucene.index.SegmentMerger.mergeTerms(SegmentMerger.java:193)
   [junit4]    > 	at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:95)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4075)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3655)
   [junit4]    > 	at org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:588)
   [junit4]    > 	at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:626)
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/test/J0/temp/lucene.index.TestDuelingCodecsAtNight_509A95B358E09389-001
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene54): {}, docValues:{}, sim=DefaultSimilarity, locale=ar_EG, timezone=Indian/Maldives
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_80 (64-bit)/cpus=4,threads=1,free=352965168,total=450363392
   [junit4]   2> NOTE: All tests run in this JVM: [TestAttributeSource, TestDateFilter, TestPrefixInBooleanQuery, TestTimeLimitingCollector, TestTimSorter, Test2BPagedBytes, TestNumericRangeQuery64, TestSpanNearQuery, TestCollectionUtil, TestIndexWriterNRTIsCurrent, TestMultiPhraseQuery, TestSegmentReader, TestNumericDocValuesUpdates, Test4GBStoredFields, Test2BPositions, TestReusableStringReader, TestIndexWriterDelete, TestDocIdSetBuilder, TestDocsAndPositions, TestTieredMergePolicy, TestNoDeletionPolicy, TestSizeBoundedForceMerge, TestRoaringDocIdSet, Test2BPostingsBytes, TestManyFields, TestBagOfPositions, TestStressNRT, TestFieldReuse, TestCustomSearcherSort, TestBytesRefHash, TestMultiThreadTermVectors, TestMultiLevelSkipList, TestMixedCodecs, TestNGramPhraseQuery, TestSortedNumericSortField, TestPersistentSnapshotDeletionPolicy, TestPerSegmentDeletes, TestSimilarity2, TestForUtil, TestSumDocFreq, TestSortRandom, TestSloppyPhraseQuery2, TestSleepingLockWrapper, TestFilterSpans, TestPrefixCodedTerms, TestLucene50FieldInfoFormat, TestSimilarityProvider, TestDeletionPolicy, TestTermsEnum, TestSearcherManager, TestReaderClosed, TestNumericUtils, TestConjunctionDISI, TestQueryCachingPolicy, TestOrdinalMap, TestNot, TestSpanTermQuery, TestConjunctions, TestDemo, TestLRUQueryCache, TestRegexpRandom, TestTragicIndexWriterDeadlock, TestSubScorerFreqs, TestSimpleAttributeImpl, TestMultiFields, TestField, TestIndexWriterCommit, TestFileSwitchDirectory, TestPackedInts, TestDuelingCodecs, TestBoolean2, TestAtomicUpdate, TestNorms, TestDirectoryReaderReopen, TestSearchAfter, TestConcurrentMergeScheduler, TestHighCompressionMode, TestDocumentsWriterStallControl, TestBlockPostingsFormat3, TestLucene50SegmentInfoFormat, TestLucene50StoredFieldsFormatHighCompression, TestLucene53NormsFormat, TestDocInverterPerFieldErrorInfo, TestDuelingCodecsAtNight]
   [junit4] Completed [412/412] on J0 in 1067.17s, 3 tests, 1 error <<< FAILURES!

[...truncated 1 lines...]
   [junit4] JVM J0: stdout was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/test/temp/junit4-J0-20151123_202723_803.sysout
   [junit4] >>> JVM J0: stdout (verbatim) ----
   [junit4] java.lang.OutOfMemoryError: Java heap space
   [junit4] Dumping heap to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/heapdumps/java_pid23371.hprof ...
   [junit4] Heap dump file created [423897737 bytes in 0.732 secs]
   [junit4] <<< JVM J0: EOF ----

[...truncated 10 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build.xml:50: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 412 suites (1 ignored), 3337 tests, 1 error, 25 ignored (21 assumptions) [seed: 509A95B358E09389]

Total time: 64 minutes 26 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1025 - Still Failing

[Apache Jenkins Server <je...@builds.apache.org>]

Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1025/

1 tests failed.
FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=12979, name=collection5, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=12979, name=collection5, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:45109/xton/f: Could not find collection : awholynewstresscollection_collection5_2
	at __randomizedtesting.SeedInfo.seed([11B33928E1F9FE1A]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:574)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:240)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:229)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:883)




Build Log:
[...truncated 10748 lines...]
   [junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_11B33928E1F9FE1A-001/init-core-data-001
   [junit4]   2> 1326314 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[11B33928E1F9FE1A]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (false)
   [junit4]   2> 1326314 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[11B33928E1F9FE1A]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /xton/f
   [junit4]   2> 1326319 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 1326320 INFO  (Thread-7980) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1326320 INFO  (Thread-7980) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 1326420 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.ZkTestServer start zk server on port:41216
   [junit4]   2> 1326420 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1326420 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1326424 INFO  (zkCallback-818-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@93f6910 name:ZooKeeperConnection Watcher:127.0.0.1:41216 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1326424 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1326425 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1326425 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 1326428 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1326429 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1326430 INFO  (zkCallback-819-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6de97fb name:ZooKeeperConnection Watcher:127.0.0.1:41216/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1326430 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1326430 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1326430 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 1326432 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 1326434 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 1326436 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 1326438 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 1326438 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 1326440 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 1326441 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 1326442 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1326443 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1326444 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 1326444 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 1326446 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 1326447 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 1326448 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 1326449 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 1326451 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 1326451 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 1326453 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1326453 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1326455 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1326456 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1326457 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 1326457 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 1326459 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 1326460 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 1326462 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1326462 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1326463 INFO  (zkCallback-820-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@124f17d9 name:ZooKeeperConnection Watcher:127.0.0.1:41216/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1326463 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1326463 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1326465 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1326465 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1326466 INFO  (zkCallback-821-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@25fa1c8a name:ZooKeeperConnection Watcher:127.0.0.1:41216 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1326466 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1326467 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1326467 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 1326468 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1326469 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1326470 INFO  (zkCallback-822-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7c7a0c53 name:ZooKeeperConnection Watcher:127.0.0.1:41216/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1326470 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1326470 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1326470 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf2/solrconfig.xml
   [junit4]   2> 1326471 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/solrconfig.xml
   [junit4]   2> 1326477 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf2/schema.xml
   [junit4]   2> 1326477 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/schema.xml
   [junit4]   2> 1326479 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf2/enumsConfig.xml
   [junit4]   2> 1326479 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/enumsConfig.xml
   [junit4]   2> 1326481 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf2/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1326481 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1326484 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf2/stopwords.txt
   [junit4]   2> 1326484 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/stopwords.txt
   [junit4]   2> 1326485 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf2/protwords.txt
   [junit4]   2> 1326485 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/protwords.txt
   [junit4]   2> 1326487 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf2/currency.xml
   [junit4]   2> 1326487 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/currency.xml
   [junit4]   2> 1326488 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf2/open-exchange-rates.json
   [junit4]   2> 1326488 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/open-exchange-rates.json
   [junit4]   2> 1326490 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf2/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1326490 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1326491 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf2/old_synonyms.txt
   [junit4]   2> 1326491 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/old_synonyms.txt
   [junit4]   2> 1326493 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf2/synonyms.txt
   [junit4]   2> 1326493 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/synonyms.txt
   [junit4]   2> 1326494 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/elevate.xml to /configs/conf2/elevate.xml
   [junit4]   2> 1326494 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/elevate.xml
   [junit4]   2> 1326787 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_11B33928E1F9FE1A-001/control-001/cores/collection1
   [junit4]   2> 1326788 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 1326790 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@394400b5{/xton/f,null,AVAILABLE}
   [junit4]   2> 1326800 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.e.j.s.ServerConnector Started ServerConnector@4004048b{SSL-http/1.1}{127.0.0.1:51454}
   [junit4]   2> 1326800 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.e.j.s.Server Started @1329811ms
   [junit4]   2> 1326800 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_11B33928E1F9FE1A-001/control-001/cores, hostContext=/xton/f, hostPort=51454, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_11B33928E1F9FE1A-001/tempDir-001/control/data}
   [junit4]   2> 1326800 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@7b3cb2c6
   [junit4]   2> 1326801 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_11B33928E1F9FE1A-001/control-001'
   [junit4]   2> 1326801 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1326801 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1326801 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1326802 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1326803 INFO  (zkCallback-823-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@74bdcb8d name:ZooKeeperConnection Watcher:127.0.0.1:41216/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1326803 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1326803 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1326804 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1326804 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_11B33928E1F9FE1A-001/control-001/solr.xml
   [junit4]   2> 1326819 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_11B33928E1F9FE1A-001/control-001/cores
   [junit4]   2> 1326819 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.CoreContainer New CoreContainer 1839779241
   [junit4]   2> 1326819 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_11B33928E1F9FE1A-001/control-001]
   [junit4]   2> 1326819 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_11B33928E1F9FE1A-001/control-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_11B33928E1F9FE1A-001/control-001/lib
   [junit4]   2> 1326819 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1326822 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1326824 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1326824 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1326825 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:41216/solr
   [junit4]   2> 1326825 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 1326825 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1326825 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1326827 INFO  (zkCallback-825-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@11b1f65 name:ZooKeeperConnection Watcher:127.0.0.1:41216 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1326827 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1326827 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1326829 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [n:127.0.0.1:51454_xton%2Ff    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1326830 INFO  (zkCallback-826-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@72e7ee name:ZooKeeperConnection Watcher:127.0.0.1:41216/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1326830 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [n:127.0.0.1:51454_xton%2Ff    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1326832 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [n:127.0.0.1:51454_xton%2Ff    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 1326834 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [n:127.0.0.1:51454_xton%2Ff    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 1326839 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [n:127.0.0.1:51454_xton%2Ff    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 1326841 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [n:127.0.0.1:51454_xton%2Ff    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 1326843 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [n:127.0.0.1:51454_xton%2Ff    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 1326846 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [n:127.0.0.1:51454_xton%2Ff    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 1326847 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [n:127.0.0.1:51454_xton%2Ff    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 1326848 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [n:127.0.0.1:51454_xton%2Ff    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 1326850 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [n:127.0.0.1:51454_xton%2Ff    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 1326851 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [n:127.0.0.1:51454_xton%2Ff    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:51454_xton%2Ff
   [junit4]   2> 1326851 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [n:127.0.0.1:51454_xton%2Ff    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:51454_xton%2Ff
   [junit4]   2> 1326853 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [n:127.0.0.1:51454_xton%2Ff    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 1326854 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [n:127.0.0.1:51454_xton%2Ff    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 1326856 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [n:127.0.0.1:51454_xton%2Ff    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1326857 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [n:127.0.0.1:51454_xton%2Ff    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94911447009853447-127.0.0.1:51454_xton%2Ff-n_0000000000
   [junit4]   2> 1326857 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [n:127.0.0.1:51454_xton%2Ff    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:51454_xton%2Ff
   [junit4]   2> 1326857 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [n:127.0.0.1:51454_xton%2Ff    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 1326859 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [n:127.0.0.1:51454_xton%2Ff    ] o.a.s.c.Overseer Overseer (id=94911447009853447-127.0.0.1:51454_xton%2Ff-n_0000000000) starting
   [junit4]   2> 1326861 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [n:127.0.0.1:51454_xton%2Ff    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 1326867 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [n:127.0.0.1:51454_xton%2Ff    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 1326868 INFO  (OverseerCollectionConfigSetProcessor-94911447009853447-127.0.0.1:51454_xton%2Ff-n_0000000000) [n:127.0.0.1:51454_xton%2Ff    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 1326868 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [n:127.0.0.1:51454_xton%2Ff    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1326868 INFO  (OverseerStateUpdate-94911447009853447-127.0.0.1:51454_xton%2Ff-n_0000000000) [n:127.0.0.1:51454_xton%2Ff    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 1326886 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [n:127.0.0.1:51454_xton%2Ff    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1326886 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [n:127.0.0.1:51454_xton%2Ff    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1326887 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [n:127.0.0.1:51454_xton%2Ff    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_11B33928E1F9FE1A-001/control-001/cores
   [junit4]   2> 1326888 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [n:127.0.0.1:51454_xton%2Ff    ] o.a.s.c.SolrCore Created CoreDescriptor: {configSetProperties=configsetprops.json, dataDir=data/, name=collection1, collection=control_collection, schema=schema.xml, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_11B33928E1F9FE1A-001/control-001/cores/collection1, loadOnStartup=true, config=solrconfig.xml, coreNodeName=, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_11B33928E1F9FE1A-001/control-001/cores/collection1/, transient=false, shard=}
   [junit4]   2> 1326888 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [n:127.0.0.1:51454_xton%2Ff    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_11B33928E1F9FE1A-001/control-001/cores/collection1/
   [junit4]   2> 1326888 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [n:127.0.0.1:51454_xton%2Ff    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 1326889 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [n:127.0.0.1:51454_xton%2Ff    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 1326889 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [n:127.0.0.1:51454_xton%2Ff    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1326890 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 1326890 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1326891 INFO  (OverseerStateUpdate-94911447009853447-127.0.0.1:51454_xton%2Ff-n_0000000000) [n:127.0.0.1:51454_xton%2Ff    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "node_name":"127.0.0.1:51454_xton%2Ff",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "shard":null,
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "base_url":"https://127.0.0.1:51454/xton/f",
   [junit4]   2>   "collection":"control_collection"} current state version: 0
   [junit4]   2> 1326892 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 1326892 INFO  (OverseerStateUpdate-94911447009853447-127.0.0.1:51454_xton%2Ff-n_0000000000) [n:127.0.0.1:51454_xton%2Ff    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "node_name":"127.0.0.1:51454_xton%2Ff",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "shard":null,
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "base_url":"https://127.0.0.1:51454/xton/f",
   [junit4]   2>   "collection":"control_collection"}
   [junit4]   2> 1326892 INFO  (OverseerStateUpdate-94911447009853447-127.0.0.1:51454_xton%2Ff-n_0000000000) [n:127.0.0.1:51454_xton%2Ff    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 1326893 INFO  (OverseerStateUpdate-94911447009853447-127.0.0.1:51454_xton%2Ff-n_0000000000) [n:127.0.0.1:51454_xton%2Ff    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 1326896 INFO  (zkCallback-826-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1327892 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1327892 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 1327893 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 1327893 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 1327894 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1327894 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_11B33928E1F9FE1A-001/control-001/cores/collection1'
   [junit4]   2> 1327894 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1327894 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1327896 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 1327902 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 1327930 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 1327960 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1327971 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1328266 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 1328274 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 1328276 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 1328285 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1328304 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1328308 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1328313 WARN  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 1328313 WARN  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 1328313 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1328314 WARN  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 1328314 WARN  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 1328315 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 1328316 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 1328316 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 1328316 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_11B33928E1F9FE1A-001/control-001/cores/collection1], dataDir=[null]
   [junit4]   2> 1328316 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@3e379080
   [junit4]   2> 1328317 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_11B33928E1F9FE1A-001/control-001/cores/collection1/data
   [junit4]   2> 1328317 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_11B33928E1F9FE1A-001/control-001/cores/collection1/data/index/
   [junit4]   2> 1328317 WARN  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_11B33928E1F9FE1A-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 1328317 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_11B33928E1F9FE1A-001/control-001/cores/collection1/data/index
   [junit4]   2> 1328317 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=17, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
   [junit4]   2> 1328318 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@259a5a5c lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@38a2614d),segFN=segments_1,generation=1}
   [junit4]   2> 1328318 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 1328324 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 1328324 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 1328324 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 1328324 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1328324 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1328325 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 1328325 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1328325 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1328325 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 1328325 WARN  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
   [junit4]   2> 1328328 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1328329 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1328331 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1328332 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1328332 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_11B33928E1F9FE1A-001/control-001/cores/collection1/data/
   [junit4]   2> 1328332 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_11B33928E1F9FE1A-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 1328337 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /update/json,/admin/file,/admin/mbeans,/replication,/admin/logging,/update,standard,/admin/plugins,/config,/admin/ping,/dump,/update/json/docs,/get,/admin/threads,/admin/segments,/update/csv,/admin/system,/admin/properties,/schema,/admin/luke
   [junit4]   2> 1328338 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 1328346 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1328346 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1328347 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 1328347 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 1328347 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=3, maxMergeAtOnceExplicit=4, maxMergedSegmentMB=87.654296875, floorSegmentMB=1.4326171875, forceMergeDeletesPctAllowed=10.480213125554982, segmentsPerTier=4.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.20849498107086886
   [junit4]   2> 1328348 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@259a5a5c lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@38a2614d),segFN=segments_1,generation=1}
   [junit4]   2> 1328348 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 1328348 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@2da091f1[collection1] main
   [junit4]   2> 1328348 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 1328353 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1328353 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 1328353 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 1328353 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 1328354 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1328354 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 1328354 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1328354 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 1328354 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1328355 INFO  (searcherExecutor-3342-thread-1-processing-x:collection1 n:127.0.0.1:51454_xton%2Ff r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@2da091f1[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1328355 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 1328355 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 1328355 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 1328355 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1518583154290458624
   [junit4]   2> 1328358 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 2.0ms to seed version buckets with highest version 1518583154290458624
   [junit4]   2> 1328358 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 1328358 INFO  (coreLoadExecutor-3341-thread-1-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 1328364 INFO  (coreZkRegister-3335-thread-1-processing-x:collection1 n:127.0.0.1:51454_xton%2Ff r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:https://127.0.0.1:51454/xton/f collection:control_collection shard:shard1
   [junit4]   2> 1328365 INFO  (coreZkRegister-3335-thread-1-processing-x:collection1 n:127.0.0.1:51454_xton%2Ff r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 1328370 INFO  (coreZkRegister-3335-thread-1-processing-x:collection1 n:127.0.0.1:51454_xton%2Ff r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94911447009853447-core_node1-n_0000000000
   [junit4]   2> 1328371 INFO  (coreZkRegister-3335-thread-1-processing-x:collection1 n:127.0.0.1:51454_xton%2Ff r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 1328372 INFO  (coreZkRegister-3335-thread-1-processing-x:collection1 n:127.0.0.1:51454_xton%2Ff r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1328372 INFO  (coreZkRegister-3335-thread-1-processing-x:collection1 n:127.0.0.1:51454_xton%2Ff r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1328372 INFO  (coreZkRegister-3335-thread-1-processing-x:collection1 n:127.0.0.1:51454_xton%2Ff r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:51454/xton/f/collection1/
   [junit4]   2> 1328372 INFO  (coreZkRegister-3335-thread-1-processing-x:collection1 n:127.0.0.1:51454_xton%2Ff r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 1328372 INFO  (coreZkRegister-3335-thread-1-processing-x:collection1 n:127.0.0.1:51454_xton%2Ff r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy https://127.0.0.1:51454/xton/f/collection1/ has no replicas
   [junit4]   2> 1328373 INFO  (OverseerStateUpdate-94911447009853447-127.0.0.1:51454_xton%2Ff-n_0000000000) [n:127.0.0.1:51454_xton%2Ff    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 1328374 INFO  (coreZkRegister-3335-thread-1-processing-x:collection1 n:127.0.0.1:51454_xton%2Ff r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 1328376 INFO  (coreZkRegister-3335-thread-1-processing-x:collection1 n:127.0.0.1:51454_xton%2Ff r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 1328378 INFO  (coreZkRegister-3335-thread-1-processing-x:collection1 n:127.0.0.1:51454_xton%2Ff r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:51454/xton/f/collection1/ shard1
   [junit4]   2> 1328378 INFO  (OverseerStateUpdate-94911447009853447-127.0.0.1:51454_xton%2Ff-n_0000000000) [n:127.0.0.1:51454_xton%2Ff    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"https://127.0.0.1:51454/xton/f",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 1
   [junit4]   2> 1328394 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1328396 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1328396 INFO  (zkCallback-828-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@294ea488 name:ZooKeeperConnection Watcher:127.0.0.1:41216/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1328397 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1328397 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1328397 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1328399 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 1328399 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with stateFormat=2
   [junit4]   2> 1328399 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1328405 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1328405 INFO  (zkCallback-829-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@17ac3a71 name:ZooKeeperConnection Watcher:127.0.0.1:41216/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1328406 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1328406 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1328408 INFO  (OverseerStateUpdate-94911447009853447-127.0.0.1:51454_xton%2Ff-n_0000000000) [n:127.0.0.1:51454_xton%2Ff    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"create",
   [junit4]   2>   "name":"collection1",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "stateFormat":"2"} current state version: 1
   [junit4]   2> 1328408 INFO  (OverseerStateUpdate-94911447009853447-127.0.0.1:51454_xton%2Ff-n_0000000000) [n:127.0.0.1:51454_xton%2Ff    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 1328409 INFO  (zkCallback-826-thread-3-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1328412 INFO  (zkCallback-828-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1328429 INFO  (coreZkRegister-3335-thread-1-processing-x:collection1 n:127.0.0.1:51454_xton%2Ff r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are https://127.0.0.1:51454/xton/f/collection1/ and leader is https://127.0.0.1:51454/xton/f/collection1/
   [junit4]   2> 1328429 INFO  (coreZkRegister-3335-thread-1-processing-x:collection1 n:127.0.0.1:51454_xton%2Ff r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:51454/xton/f
   [junit4]   2> 1328429 INFO  (coreZkRegister-3335-thread-1-processing-x:collection1 n:127.0.0.1:51454_xton%2Ff r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 1328429 INFO  (coreZkRegister-3335-thread-1-processing-x:collection1 n:127.0.0.1:51454_xton%2Ff r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 1328429 INFO  (coreZkRegister-3335-thread-1-processing-x:collection1 n:127.0.0.1:51454_xton%2Ff r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:51454_xton%2Ff c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1328436 INFO  (OverseerStateUpdate-94911447009853447-127.0.0.1:51454_xton%2Ff-n_0000000000) [n:127.0.0.1:51454_xton%2Ff    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "node_name":"127.0.0.1:51454_xton%2Ff",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "base_url":"https://127.0.0.1:51454/xton/f",
   [junit4]   2>   "collection":"control_collection"} current state version: 2
   [junit4]   2> 1328437 INFO  (OverseerStateUpdate-94911447009853447-127.0.0.1:51454_xton%2Ff-n_0000000000) [n:127.0.0.1:51454_xton%2Ff    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "node_name":"127.0.0.1:51454_xton%2Ff",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "base_url":"https://127.0.0.1:51454/xton/f",
   [junit4]   2>   "collection":"control_collection"}
   [junit4]   2> 1328442 INFO  (OverseerStateUpdate-94911447009853447-127.0.0.1:51454_xton%2Ff-n_0000000000) [n:127.0.0.1:51454_xton%2Ff    ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/collection1/state.json
   [junit4]   2> 1328552 INFO  (zkCallback-828-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1328552 INFO  (zkCallback-826-thread-3-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1328721 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_11B33928E1F9FE1A-001/shard-1-001/cores/collection1
   [junit4]   2> 1328722 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_11B33928E1F9FE1A-001/shard-1-001
   [junit4]   2> 1328723 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 1328725 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@87f3720{/xton/f,null,AVAILABLE}
   [junit4]   2> 1328734 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.e.j.s.ServerConnector Started ServerConnector@7c8d7f0b{SSL-http/1.1}{127.0.0.1:53114}
   [junit4]   2> 1328735 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.e.j.s.Server Started @1331745ms
   [junit4]   2> 1328735 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solrconfig=solrconfig.xml, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_11B33928E1F9FE1A-001/tempDir-001/jetty1, hostContext=/xton/f, hostPort=53114, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_11B33928E1F9FE1A-001/shard-1-001/cores}
   [junit4]   2> 1328735 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@7b3cb2c6
   [junit4]   2> 1328735 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_11B33928E1F9FE1A-001/shard-1-001'
   [junit4]   2> 1328735 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1328735 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1328736 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1328737 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1328738 INFO  (zkCallback-830-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@498aea07 name:ZooKeeperConnection Watcher:127.0.0.1:41216/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1328738 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1328738 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1328739 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1328740 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_11B33928E1F9FE1A-001/shard-1-001/solr.xml
   [junit4]   2> 1328755 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_11B33928E1F9FE1A-001/shard-1-001/cores
   [junit4]   2> 1328755 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.CoreContainer New CoreContainer 1234197051
   [junit4]   2> 1328755 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_11B33928E1F9FE1A-001/shard-1-001]
   [junit4]   2> 1328755 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_11B33928E1F9FE1A-001/shard-1-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_11B33928E1F9FE1A-001/shard-1-001/lib
   [junit4]   2> 1328756 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1328759 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1328761 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1328761 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1328761 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:41216/solr
   [junit4]   2> 1328761 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 1328761 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1328764 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1328768 INFO  (zkCallback-832-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@36bd3a48 name:ZooKeeperConnection Watcher:127.0.0.1:41216 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1328769 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1328769 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1328780 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [n:127.0.0.1:53114_xton%2Ff    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1328781 INFO  (zkCallback-833-thread-1-processing-n:127.0.0.1:53114_xton%2Ff) [n:127.0.0.1:53114_xton%2Ff    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6c2d8313 name:ZooKeeperConnection Watcher:127.0.0.1:41216/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1328781 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [n:127.0.0.1:53114_xton%2Ff    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1328785 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [n:127.0.0.1:53114_xton%2Ff    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1329791 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [n:127.0.0.1:53114_xton%2Ff    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:53114_xton%2Ff
   [junit4]   2> 1329791 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [n:127.0.0.1:53114_xton%2Ff    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:53114_xton%2Ff
   [junit4]   2> 1329792 INFO  (zkCallback-826-thread-3-processing-n:127.0.0.1:51454_xton%2Ff) [n:127.0.0.1:51454_xton%2Ff    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1329792 INFO  (zkCallback-833-thread-1-processing-n:127.0.0.1:53114_xton%2Ff) [n:127.0.0.1:53114_xton%2Ff    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1329793 INFO  (zkCallback-828-thread-1) [    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1329794 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [n:127.0.0.1:53114_xton%2Ff    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1329795 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [n:127.0.0.1:53114_xton%2Ff    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94911447009853452-127.0.0.1:53114_xton%2Ff-n_0000000001
   [junit4]   2> 1329795 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [n:127.0.0.1:53114_xton%2Ff    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/94911447009853447-127.0.0.1:51454_xton%2Ff-n_0000000000 to know if I could be the leader
   [junit4]   2> 1329805 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [n:127.0.0.1:53114_xton%2Ff    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1329805 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [n:127.0.0.1:53114_xton%2Ff    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1329806 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [n:127.0.0.1:53114_xton%2Ff    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_11B33928E1F9FE1A-001/shard-1-001/cores
   [junit4]   2> 1329807 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [n:127.0.0.1:53114_xton%2Ff    ] o.a.s.c.SolrCore Created CoreDescriptor: {transient=false, configSetProperties=configsetprops.json, name=collection1, collection=collection1, shard=, loadOnStartup=true, dataDir=data/, coreNodeName=, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_11B33928E1F9FE1A-001/shard-1-001/cores/collection1/, schema=schema.xml, config=solrconfig.xml, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_11B33928E1F9FE1A-001/shard-1-001/cores/collection1}
   [junit4]   2> 1329807 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [n:127.0.0.1:53114_xton%2Ff    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_11B33928E1F9FE1A-001/shard-1-001/cores/collection1/
   [junit4]   2> 1329807 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[11B33928E1F9FE1A]) [n:127.0.0.1:53114_xton%2Ff    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 1329810 INFO  (coreLoadExecutor-3352-thread-1-processing-n:127.0.0.1:53114_xton%2Ff) [n:127.0.0.1:53114_xton%2Ff c:collection1   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 1329810 INFO  (coreLoadExecutor-3352-thread-1-processing-n:127.0.0.1:53114_xton%2Ff) [n:127.0.0.1:53114_xton%2Ff c:collection1   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1329811 INFO  (TEST-Co

[...truncated too long message...]

455-core_node20-n_0000000001
   [junit4]   2> 	2	/solr/collections/awholynewcollection_5/state.json
   [junit4]   2> 	2	/solr/collections/awholynewstresscollection_collection0_2/leader_elect/shard1/election/94911447009853467-core_node13-n_0000000000
   [junit4]   2> 	2	/solr/collections/awholynewstresscollection_collection4_6/leader_elect/shard5/election/94911447009853452-core_node2-n_0000000001
   [junit4]   2> 	2	/solr/collections/awholynewstresscollection_collection4_9/leader_elect/shard3/election/94911447009853458-core_node9-n_0000000000
   [junit4]   2> 	2	/solr/collections/awholynewstresscollection_collection1_9/leader_elect/shard3/election/94911447009853452-core_node7-n_0000000001
   [junit4]   2> 	2	/solr/collections/awholynewstresscollection_collection1_6/leader_elect/shard3/election/94911447009853455-core_node18-n_0000000001
   [junit4]   2> 	2	/solr/collections/awholynewstresscollection_collection3_6/leader_elect/shard4/election/94911447009853467-core_node17-n_0000000001
   [junit4]   2> 	2	/solr/collections/awholynewstresscollection_collection4_6/leader_elect/shard1/election/94911447009853447-core_node16-n_0000000001
   [junit4]   2> 	2	/solr/collections/awholynewstresscollection_collection4_2/leader_elect/shard4/election/94911447009853452-core_node3-n_0000000000
   [junit4]   2> 	2	/solr/collections/awholynewstresscollection_collection2_6/leader_elect/shard1/election/94911447009853455-core_node13-n_0000000001
   [junit4]   2> 	2	/solr/collections/awholynewstresscollection_collection0_2/leader_elect/shard2/election/94911447009853452-core_node6-n_0000000000
   [junit4]   2> 	2	/solr/collections/awholynewstresscollection_collection3_5/leader_elect/shard1/election/94911447009853455-core_node3-n_0000000000
   [junit4]   2> 	2	/solr/collections/awholynewstresscollection_collection3_9/leader_elect/shard1/election/94911447009853458-core_node2-n_0000000001
   [junit4]   2> 
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2> 
   [junit4]   2> 	62	/solr/collections
   [junit4]   2> 	61	/solr/live_nodes
   [junit4]   2> 	3	/solr/overseer/queue
   [junit4]   2> 	3	/solr/overseer/collection-queue-work
   [junit4]   2> 
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=CollectionsAPIDistributedZkTest -Dtests.method=test -Dtests.seed=11B33928E1F9FE1A -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=it -Dtests.timezone=Asia/Urumqi -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR    506s J2 | CollectionsAPIDistributedZkTest.test <<<
   [junit4]    > Throwable #1: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=12979, name=collection5, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    > Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:45109/xton/f: Could not find collection : awholynewstresscollection_collection5_2
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([11B33928E1F9FE1A]:0)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:574)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:240)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:229)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:883)Throwable #2: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=12974, name=collection0, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    > Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:45109/xton/f: Could not find collection : awholynewstresscollection_collection0_2
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([11B33928E1F9FE1A]:0)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:574)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:240)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:229)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:883)
   [junit4]   2> 1832794 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[11B33928E1F9FE1A]-worker) [n:127.0.0.1:48850_xton%2Ff c:addReplicaColl s:shard1 r:core_node7 x:propertyDotName] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: test params are: codec=SimpleText, sim=DefaultSimilarity, locale=it, timezone=Asia/Urumqi
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_80 (64-bit)/cpus=4,threads=1,free=242498288,total=478150656
   [junit4]   2> NOTE: All tests run in this JVM: [TestPKIAuthenticationPlugin, AutoCommitTest, DataDrivenBlockJoinTest, TestSolr4Spatial2, SuggesterTSTTest, TestSearchPerf, CoreAdminHandlerTest, PluginInfoTest, TestCoreDiscovery, TestMiniSolrCloudCluster, TestCollationField, TestSearcherReuse, SolrCloudExampleTest, LukeRequestHandlerTest, StatelessScriptUpdateProcessorFactoryTest, DateMathParserTest, QueryParsingTest, TestUtils, TestReloadDeadlock, ClusterStateTest, TestSchemaSimilarityResource, TestMacros, TestRandomMergePolicy, TestSerializedLuceneMatchVersion, CollectionStateFormat2Test, TestManagedStopFilterFactory, CoreMergeIndexesAdminHandlerTest, ShowFileRequestHandlerTest, DistribDocExpirationUpdateProcessorTest, TestInitQParser, TestTolerantSearch, TestSolrConfigHandler, HdfsThreadLeakTest, TestStressReorder, StressHdfsTest, TestExpandComponent, TestSha256AuthenticationProvider, DeleteLastCustomShardedReplicaTest, CurrencyFieldXmlFileTest, TestExtendedDismaxParser, TestReversedWildcardFilterFactory, TestJsonFacets, XmlUpdateRequestHandlerTest, DistributedQueryComponentCustomSortTest, HdfsNNFailoverTest, TestReload, UpdateParamsTest, SuggesterTest, DeleteReplicaTest, OverseerTaskQueueTest, DistributedQueueTest, TestCustomDocTransformer, TestHdfsUpdateLog, TestJmxMonitoredMap, TestManagedResourceStorage, ZkStateReaderTest, LeaderElectionTest, FullSolrCloudDistribCmdsTest, SolrCoreCheckLockOnStartupTest, TestRawTransformer, ConnectionManagerTest, TestExactStatsCache, TestSolrDeletionPolicy1, BinaryUpdateRequestHandlerTest, FieldMutatingUpdateProcessorTest, BasicDistributedZk2Test, PrimitiveFieldTypeTest, HLLUtilTest, DistributedFacetPivotSmallAdvancedTest, TestWriterPerf, SSLMigrationTest, TestZkChroot, AnalyticsMergeStrategyTest, DirectUpdateHandlerOptimizeTest, TestDocBasedVersionConstraints, TestRecoveryHdfs, DistributedVersionInfoTest, RegexBoostProcessorTest, TestPHPSerializedResponseWriter, DistanceUnitsTest, HighlighterMaxOffsetTest, HdfsRecoveryZkTest, UnloadDistributedZkTest, ScriptEngineTest, TestSolrQueryParserResource, CloudExitableDirectoryReaderTest, CollectionsAPIDistributedZkTest]
   [junit4] Completed [204/549] on J2 in 507.22s, 1 test, 1 error <<< FAILURES!

[...truncated 1134 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 549 suites (7 ignored), 2166 tests, 1 error, 32 ignored (18 assumptions) [seed: 11B33928E1F9FE1A]

Total time: 204 minutes 34 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1024 - Still Failing

[Apache Jenkins Server <je...@builds.apache.org>]

Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1024/

2 tests failed.
FAILED:  org.apache.solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR

Error Message:
There are still nodes recoverying - waited for 330 seconds

Stack Trace:
java.lang.AssertionError: There are still nodes recoverying - waited for 330 seconds
	at __randomizedtesting.SeedInfo.seed([99E29040E82FC4E0:4768FE3DD38DA613]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:170)
	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:131)
	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:126)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.waitForRecoveriesToFinish(AbstractFullDistribZkTestBase.java:830)
	at org.apache.solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR(LeaderInitiatedRecoveryOnShardRestartTest.java:109)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:963)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:938)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=15326, name=collection2, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=15326, name=collection2, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:59542/da_gh: collection already exists: awholynewstresscollection_collection2_0
	at __randomizedtesting.SeedInfo.seed([99E29040E82FC4E0]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1573)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1594)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:876)




Build Log:
[...truncated 10995 lines...]
   [junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_99E29040E82FC4E0-001/init-core-data-001
   [junit4]   2> 1779632 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[99E29040E82FC4E0]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true)
   [junit4]   2> 1779632 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[99E29040E82FC4E0]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /da_gh/
   [junit4]   2> 1779637 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 1779637 INFO  (Thread-9737) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1779637 INFO  (Thread-9737) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 1779737 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.ZkTestServer start zk server on port:54804
   [junit4]   2> 1779737 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1779738 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1779740 INFO  (zkCallback-1094-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@66e3a7c2 name:ZooKeeperConnection Watcher:127.0.0.1:54804 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1779741 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1779741 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1779741 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 1779744 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1779745 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1779746 INFO  (zkCallback-1095-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@25ac1c59 name:ZooKeeperConnection Watcher:127.0.0.1:54804/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1779746 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1779746 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1779746 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 1779748 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 1779750 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 1779751 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 1779753 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 1779753 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 1779756 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 1779756 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 1779758 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1779759 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1779760 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 1779761 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 1779762 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 1779763 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 1779764 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 1779765 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 1779766 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 1779767 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 1779768 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1779768 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1779770 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1779770 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1779772 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 1779772 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 1779774 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 1779775 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 1780121 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_99E29040E82FC4E0-001/control-001/cores/collection1
   [junit4]   2> 1780122 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 1780124 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@7b2f5fe8{/da_gh,null,AVAILABLE}
   [junit4]   2> 1780127 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.e.j.s.ServerConnector Started ServerConnector@4f177300{HTTP/1.1}{127.0.0.1:54698}
   [junit4]   2> 1780127 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.e.j.s.Server Started @1782908ms
   [junit4]   2> 1780127 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/da_gh, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_99E29040E82FC4E0-001/control-001/cores, hostPort=54698, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_99E29040E82FC4E0-001/tempDir-001/control/data}
   [junit4]   2> 1780127 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@7b3cb2c6
   [junit4]   2> 1780127 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_99E29040E82FC4E0-001/control-001'
   [junit4]   2> 1780128 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1780128 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1780128 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1780128 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1780130 INFO  (zkCallback-1096-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@40cb6525 name:ZooKeeperConnection Watcher:127.0.0.1:54804/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1780130 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1780130 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1780131 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1780131 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_99E29040E82FC4E0-001/control-001/solr.xml
   [junit4]   2> 1780145 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_99E29040E82FC4E0-001/control-001/cores
   [junit4]   2> 1780145 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.CoreContainer New CoreContainer 44070698
   [junit4]   2> 1780146 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_99E29040E82FC4E0-001/control-001]
   [junit4]   2> 1780146 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_99E29040E82FC4E0-001/control-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_99E29040E82FC4E0-001/control-001/lib
   [junit4]   2> 1780146 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1780147 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1780148 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1780148 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1780148 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:54804/solr
   [junit4]   2> 1780148 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 1780148 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1780148 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1780150 INFO  (zkCallback-1098-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@20214727 name:ZooKeeperConnection Watcher:127.0.0.1:54804 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1780151 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1780151 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1780153 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [n:127.0.0.1:54698_da_gh    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1780154 INFO  (zkCallback-1099-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@33a9c8e4 name:ZooKeeperConnection Watcher:127.0.0.1:54804/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1780154 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [n:127.0.0.1:54698_da_gh    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1780155 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [n:127.0.0.1:54698_da_gh    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 1780158 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [n:127.0.0.1:54698_da_gh    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 1780163 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [n:127.0.0.1:54698_da_gh    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 1780166 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [n:127.0.0.1:54698_da_gh    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 1780168 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [n:127.0.0.1:54698_da_gh    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 1780170 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [n:127.0.0.1:54698_da_gh    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 1780172 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [n:127.0.0.1:54698_da_gh    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 1780173 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [n:127.0.0.1:54698_da_gh    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 1780174 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [n:127.0.0.1:54698_da_gh    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 1780175 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [n:127.0.0.1:54698_da_gh    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:54698_da_gh
   [junit4]   2> 1780176 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [n:127.0.0.1:54698_da_gh    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:54698_da_gh
   [junit4]   2> 1780177 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [n:127.0.0.1:54698_da_gh    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 1780178 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [n:127.0.0.1:54698_da_gh    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 1780179 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [n:127.0.0.1:54698_da_gh    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1780180 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [n:127.0.0.1:54698_da_gh    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94905938134827012-127.0.0.1:54698_da_gh-n_0000000000
   [junit4]   2> 1780180 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [n:127.0.0.1:54698_da_gh    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:54698_da_gh
   [junit4]   2> 1780180 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [n:127.0.0.1:54698_da_gh    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 1780182 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [n:127.0.0.1:54698_da_gh    ] o.a.s.c.Overseer Overseer (id=94905938134827012-127.0.0.1:54698_da_gh-n_0000000000) starting
   [junit4]   2> 1780183 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [n:127.0.0.1:54698_da_gh    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 1780189 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [n:127.0.0.1:54698_da_gh    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 1780190 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [n:127.0.0.1:54698_da_gh    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1780190 INFO  (OverseerCollectionConfigSetProcessor-94905938134827012-127.0.0.1:54698_da_gh-n_0000000000) [n:127.0.0.1:54698_da_gh    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 1780190 INFO  (OverseerStateUpdate-94905938134827012-127.0.0.1:54698_da_gh-n_0000000000) [n:127.0.0.1:54698_da_gh    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 1780208 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [n:127.0.0.1:54698_da_gh    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1780208 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [n:127.0.0.1:54698_da_gh    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1780209 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [n:127.0.0.1:54698_da_gh    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_99E29040E82FC4E0-001/control-001/cores
   [junit4]   2> 1780210 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [n:127.0.0.1:54698_da_gh    ] o.a.s.c.SolrCore Created CoreDescriptor: {loadOnStartup=true, coreNodeName=, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_99E29040E82FC4E0-001/control-001/cores/collection1, config=solrconfig.xml, configSetProperties=configsetprops.json, transient=false, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_99E29040E82FC4E0-001/control-001/cores/collection1/, dataDir=data/, schema=schema.xml, shard=, name=collection1, collection=control_collection}
   [junit4]   2> 1780210 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [n:127.0.0.1:54698_da_gh    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_99E29040E82FC4E0-001/control-001/cores/collection1/
   [junit4]   2> 1780211 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [n:127.0.0.1:54698_da_gh    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 1780212 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [n:127.0.0.1:54698_da_gh    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0
   [junit4]   2> 1780212 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [n:127.0.0.1:54698_da_gh    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1780212 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 1780212 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1780214 INFO  (OverseerStateUpdate-94905938134827012-127.0.0.1:54698_da_gh-n_0000000000) [n:127.0.0.1:54698_da_gh    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "node_name":"127.0.0.1:54698_da_gh",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "base_url":"http://127.0.0.1:54698/da_gh",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core":"collection1"} current state version: 0
   [junit4]   2> 1780215 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 1780215 INFO  (OverseerStateUpdate-94905938134827012-127.0.0.1:54698_da_gh-n_0000000000) [n:127.0.0.1:54698_da_gh    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "node_name":"127.0.0.1:54698_da_gh",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "base_url":"http://127.0.0.1:54698/da_gh",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core":"collection1"}
   [junit4]   2> 1780215 INFO  (OverseerStateUpdate-94905938134827012-127.0.0.1:54698_da_gh-n_0000000000) [n:127.0.0.1:54698_da_gh    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 1780216 INFO  (OverseerStateUpdate-94905938134827012-127.0.0.1:54698_da_gh-n_0000000000) [n:127.0.0.1:54698_da_gh    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 1780217 INFO  (zkCallback-1099-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1781215 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1781215 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 1781216 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 1781216 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 1781217 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1781217 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_99E29040E82FC4E0-001/control-001/cores/collection1'
   [junit4]   2> 1781217 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1781217 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1781219 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 1781225 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 1781238 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 1781285 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1781292 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1781529 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 1781536 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 1781538 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 1781547 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1781550 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1781552 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1781553 WARN  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 1781553 WARN  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 1781553 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1781554 WARN  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 1781554 WARN  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 1781555 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0
   [junit4]   2> 1781555 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 1781555 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 1781555 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_99E29040E82FC4E0-001/control-001/cores/collection1], dataDir=[null]
   [junit4]   2> 1781555 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1e15c1ec
   [junit4]   2> 1781556 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_99E29040E82FC4E0-001/control-001/cores/collection1/data
   [junit4]   2> 1781556 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_99E29040E82FC4E0-001/control-001/cores/collection1/data/index/
   [junit4]   2> 1781556 WARN  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_99E29040E82FC4E0-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 1781557 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_99E29040E82FC4E0-001/control-001/cores/collection1/data/index
   [junit4]   2> 1781557 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=8, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.43692037758476887]
   [junit4]   2> 1781557 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@12484a6 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@480aee0),segFN=segments_1,generation=1}
   [junit4]   2> 1781557 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 1781558 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_99E29040E82FC4E0-001/control-001/cores/collection1/data/
   [junit4]   2> 1781558 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_99E29040E82FC4E0-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 1781562 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 1781562 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 1781562 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 1781562 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1781563 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1781563 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 1781563 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1781563 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1781563 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 1781563 WARN  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
   [junit4]   2> 1781566 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1781567 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1781568 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1781569 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1781573 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/logging,/admin/threads,/replication,/admin/segments,/admin/plugins,/admin/ping,/dump,/config,/admin/properties,/admin/mbeans,/schema,/update/json,standard,/admin/file,/get,/update/csv,/admin/luke,/admin/system,/update/json/docs,/update
   [junit4]   2> 1781574 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 1781575 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1781575 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1781576 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 1781576 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 1781576 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=28, maxMergeAtOnceExplicit=27, maxMergedSegmentMB=63.697265625, floorSegmentMB=2.177734375, forceMergeDeletesPctAllowed=18.46726359604167, segmentsPerTier=15.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 1781577 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@12484a6 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@480aee0),segFN=segments_1,generation=1}
   [junit4]   2> 1781577 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 1781577 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@16f3fe24[collection1] main
   [junit4]   2> 1781577 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 1781578 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1781578 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 1781578 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 1781578 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 1781578 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1781579 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 1781579 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1781579 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 1781579 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1781580 INFO  (searcherExecutor-3519-thread-1-processing-n:127.0.0.1:54698_da_gh r:core_node1 s:shard1 c:control_collection x:collection1) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@16f3fe24[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1781580 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 1781580 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 1781580 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 1781580 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1518495012192518144
   [junit4]   2> 1781582 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 2.0ms to seed version buckets with highest version 1518495012192518144
   [junit4]   2> 1781582 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 1781583 INFO  (coreLoadExecutor-3518-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 1781583 INFO  (coreZkRegister-3512-thread-1-processing-n:127.0.0.1:54698_da_gh r:core_node1 s:shard1 c:control_collection x:collection1) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:54698/da_gh collection:control_collection shard:shard1
   [junit4]   2> 1781584 INFO  (coreZkRegister-3512-thread-1-processing-n:127.0.0.1:54698_da_gh r:core_node1 s:shard1 c:control_collection x:collection1) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 1781588 INFO  (coreZkRegister-3512-thread-1-processing-n:127.0.0.1:54698_da_gh r:core_node1 s:shard1 c:control_collection x:collection1) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94905938134827012-core_node1-n_0000000000
   [junit4]   2> 1781592 INFO  (coreZkRegister-3512-thread-1-processing-n:127.0.0.1:54698_da_gh r:core_node1 s:shard1 c:control_collection x:collection1) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 1781594 INFO  (coreZkRegister-3512-thread-1-processing-n:127.0.0.1:54698_da_gh r:core_node1 s:shard1 c:control_collection x:collection1) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1781594 INFO  (coreZkRegister-3512-thread-1-processing-n:127.0.0.1:54698_da_gh r:core_node1 s:shard1 c:control_collection x:collection1) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1781594 INFO  (coreZkRegister-3512-thread-1-processing-n:127.0.0.1:54698_da_gh r:core_node1 s:shard1 c:control_collection x:collection1) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:54698/da_gh/collection1/
   [junit4]   2> 1781594 INFO  (OverseerStateUpdate-94905938134827012-127.0.0.1:54698_da_gh-n_0000000000) [n:127.0.0.1:54698_da_gh    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 1781594 INFO  (coreZkRegister-3512-thread-1-processing-n:127.0.0.1:54698_da_gh r:core_node1 s:shard1 c:control_collection x:collection1) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 1781594 INFO  (coreZkRegister-3512-thread-1-processing-n:127.0.0.1:54698_da_gh r:core_node1 s:shard1 c:control_collection x:collection1) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:54698/da_gh/collection1/ has no replicas
   [junit4]   2> 1781596 INFO  (coreZkRegister-3512-thread-1-processing-n:127.0.0.1:54698_da_gh r:core_node1 s:shard1 c:control_collection x:collection1) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 1781598 INFO  (coreZkRegister-3512-thread-1-processing-n:127.0.0.1:54698_da_gh r:core_node1 s:shard1 c:control_collection x:collection1) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 1781600 INFO  (coreZkRegister-3512-thread-1-processing-n:127.0.0.1:54698_da_gh r:core_node1 s:shard1 c:control_collection x:collection1) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:54698/da_gh/collection1/ shard1
   [junit4]   2> 1781601 INFO  (OverseerStateUpdate-94905938134827012-127.0.0.1:54698_da_gh-n_0000000000) [n:127.0.0.1:54698_da_gh    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:54698/da_gh",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 1
   [junit4]   2> 1781703 INFO  (zkCallback-1099-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1781713 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1781716 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1781717 INFO  (zkCallback-1101-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@52e4cb3f name:ZooKeeperConnection Watcher:127.0.0.1:54804/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1781717 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1781717 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1781717 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1781720 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 1781720 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with stateFormat=2
   [junit4]   2> 1781720 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1781727 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1781727 INFO  (zkCallback-1102-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@37899c4a name:ZooKeeperConnection Watcher:127.0.0.1:54804/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1781728 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1781728 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1781732 INFO  (OverseerStateUpdate-94905938134827012-127.0.0.1:54698_da_gh-n_0000000000) [n:127.0.0.1:54698_da_gh    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"create",
   [junit4]   2>   "name":"collection1",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "stateFormat":"2"} current state version: 2
   [junit4]   2> 1781732 INFO  (OverseerStateUpdate-94905938134827012-127.0.0.1:54698_da_gh-n_0000000000) [n:127.0.0.1:54698_da_gh    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 1781751 INFO  (coreZkRegister-3512-thread-1-processing-n:127.0.0.1:54698_da_gh r:core_node1 s:shard1 c:control_collection x:collection1) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:54698/da_gh/collection1/ and leader is http://127.0.0.1:54698/da_gh/collection1/
   [junit4]   2> 1781752 INFO  (coreZkRegister-3512-thread-1-processing-n:127.0.0.1:54698_da_gh r:core_node1 s:shard1 c:control_collection x:collection1) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:54698/da_gh
   [junit4]   2> 1781752 INFO  (coreZkRegister-3512-thread-1-processing-n:127.0.0.1:54698_da_gh r:core_node1 s:shard1 c:control_collection x:collection1) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 1781752 INFO  (coreZkRegister-3512-thread-1-processing-n:127.0.0.1:54698_da_gh r:core_node1 s:shard1 c:control_collection x:collection1) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 1781752 INFO  (coreZkRegister-3512-thread-1-processing-n:127.0.0.1:54698_da_gh r:core_node1 s:shard1 c:control_collection x:collection1) [n:127.0.0.1:54698_da_gh c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1781755 INFO  (OverseerStateUpdate-94905938134827012-127.0.0.1:54698_da_gh-n_0000000000) [n:127.0.0.1:54698_da_gh    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "node_name":"127.0.0.1:54698_da_gh",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "base_url":"http://127.0.0.1:54698/da_gh",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core":"collection1"} current state version: 2
   [junit4]   2> 1781756 INFO  (OverseerStateUpdate-94905938134827012-127.0.0.1:54698_da_gh-n_0000000000) [n:127.0.0.1:54698_da_gh    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "node_name":"127.0.0.1:54698_da_gh",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "base_url":"http://127.0.0.1:54698/da_gh",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core":"collection1"}
   [junit4]   2> 1781756 INFO  (OverseerStateUpdate-94905938134827012-127.0.0.1:54698_da_gh-n_0000000000) [n:127.0.0.1:54698_da_gh    ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/collection1/state.json
   [junit4]   2> 1781860 INFO  (zkCallback-1099-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1781860 INFO  (zkCallback-1101-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1782032 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_99E29040E82FC4E0-001/shard-1-001/cores/collection1
   [junit4]   2> 1782033 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_99E29040E82FC4E0-001/shard-1-001
   [junit4]   2> 1782034 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 1782036 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@52b5ad1a{/da_gh,null,AVAILABLE}
   [junit4]   2> 1782036 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.e.j.s.ServerConnector Started ServerConnector@64dcfcf0{HTTP/1.1}{127.0.0.1:39166}
   [junit4]   2> 1782036 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.e.j.s.Server Started @1784817ms
   [junit4]   2> 1782036 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostPort=39166, hostContext=/da_gh, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_99E29040E82FC4E0-001/tempDir-001/jetty1, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_99E29040E82FC4E0-001/shard-1-001/cores, solrconfig=solrconfig.xml}
   [junit4]   2> 1782037 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@7b3cb2c6
   [junit4]   2> 1782037 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_99E29040E82FC4E0-001/shard-1-001'
   [junit4]   2> 1782037 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1782037 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1782037 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1782038 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1782039 INFO  (zkCallback-1103-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@10354b81 name:ZooKeeperConnection Watcher:127.0.0.1:54804/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1782039 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1782039 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1782041 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1782041 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_99E29040E82FC4E0-001/shard-1-001/solr.xml
   [junit4]   2> 1782055 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_99E29040E82FC4E0-001/shard-1-001/cores
   [junit4]   2> 1782055 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.CoreContainer New CoreContainer 29968421
   [junit4]   2> 1782055 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_99E29040E82FC4E0-001/shard-1-001]
   [junit4]   2> 1782055 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_99E29040E82FC4E0-001/shard-1-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_99E29040E82FC4E0-001/shard-1-001/lib
   [junit4]   2> 1782056 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1782057 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1782057 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1782057 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1782057 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:54804/solr
   [junit4]   2> 1782058 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 1782058 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1782058 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1782060 INFO  (zkCallback-1105-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@32e62e29 name:ZooKeeperConnection Watcher:127.0.0.1:54804 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1782060 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1782061 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1782062 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [n:127.0.0.1:39166_da_gh    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1782064 INFO  (zkCallback-1106-thread-1-processing-n:127.0.0.1:39166_da_gh) [n:127.0.0.1:39166_da_gh    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7253d80a name:ZooKeeperConnection Watcher:127.0.0.1:54804/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1782064 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [n:127.0.0.1:39166_da_gh    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1782069 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [n:127.0.0.1:39166_da_gh    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1783075 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [n:127.0.0.1:39166_da_gh    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:39166_da_gh
   [junit4]   2> 1783075 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [n:127.0.0.1:39166_da_gh    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:39166_da_gh
   [junit4]   2> 1783076 INFO  (zkCallback-1106-thread-1-processing-n:127.0.0.1:39166_da_gh) [n:127.0.0.1:39166_da_gh    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1783076 INFO  (zkCallback-1099-thread-1-processing-n:127.0.0.1:54698_da_gh) [n:127.0.0.1:54698_da_gh    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1783076 INFO  (zkCallback-1101-thread-1) [    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1783078 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [n:127.0.0.1:39166_da_gh    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1783079 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [n:127.0.0.1:39166_da_gh    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94905938134827017-127.0.0.1:39166_da_gh-n_0000000001
   [junit4]   2> 1783080 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [n:127.0.0.1:39166_da_gh    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/94905938134827012-127.0.0.1:54698_da_gh-n_0000000000 to know if I could be the leader
   [junit4]   2> 1783094 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [n:127.0.0.1:39166_da_gh    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1783094 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [n:127.0.0.1:39166_da_gh    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1783095 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [n:127.0.0.1:39166_da_gh    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_99E29040E82FC4E0-001/shard-1-001/cores
   [junit4]   2> 1783097 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [n:127.0.0.1:39166_da_gh    ] o.a.s.c.SolrCore Created CoreDescriptor: {absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_99E29040E82FC4E0-001/shard-1-001/cores/collection1/, config=solrconfig.xml, dataDir=data/, configSetProperties=configsetprops.json, coreNodeName=, loadOnStartup=true, collection=collection1, transient=false, shard=, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_99E29040E82FC4E0-001/shard-1-001/cores/collection1, name=collection1, schema=schema.xml}
   [junit4]   2> 1783097 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [n:127.0.0.1:39166_da_gh    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_99E29040E82FC4E0-001/shard-1-001/cores/collection1/
   [junit4]   2> 1783098 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [n:127.0.0.1:39166_da_gh    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 1783100 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [n:127.0.0.1:39166_da_gh    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0
   [junit4]   2> 1783100 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[99E29040E82FC4E0]) [n:127.0.0.1:39166_da_gh    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1783100 INFO  (coreLoadExecutor-3529-thread-1-processing-n:127.0.0.1:39166_da_gh) [n:127.0.0.1:39166_da_gh c:collection1   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 1783100 INFO  (coreLoadExecutor-3529-thread-1-processing-n:127.0.0.1:39166_da_gh) [n:127.0.0.1:39166_da_gh c:collection1   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1783102 INFO  (coreLoadExecutor-3529-thread-1-processing-n:127.0.0.1:39166_da_gh) [n:127.0.0.1:39166_da_gh c:collection1   x:collection1] o.a.s.c.ZkController Registering watch for collection collection1
   [junit4]   2> 1783102 INFO  (coreLoadExecutor-3529-thread-1-processing-n:127.0.0.1:39166_da_gh) [n:127.0.0.1:39166_da_gh c:collection1   x:collection1] o.a.s.c.c.ZkStateReader addZkWatch collection1
   [junit4]   2> 1783102 INFO  (OverseerStateUpdate-94905938134827012-127.0.0.1:54698_da_gh-n_0000000000) [n:127.0.0.1:54698_da_gh    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "node_name":"127.0.0.1:39166_da_gh",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "base_url":"http://127.0.0.1:39166/da_gh",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core":"collection1"} current state version: 3
   [junit4]   2> 1783103 INFO  (coreLoadExecutor-3529-thread-1-processing-n:127.0.0.1:39166_da_gh) [n:127.0.0.1:39166_da_gh c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Add data for collection1 ver 0 
   [junit4]   2> 1783103 INFO  (OverseerStateUpdate-94905938134827012-127.0.0.1:54698_da_gh-n_0000000000) [n:127.0.0.1:54698_da_gh    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "node_name":"127.0.0.1:39166_da_gh",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "base_url":"http://127.0.0.1:39166/da_gh",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core":"collection1"}
   [junit4]   2> 1783103 INFO  (coreLoadExecutor-3529-thread-1-processing-n:127.0.0.1:39166_da_gh) [n:127.0.0.1:39166_da_gh c:collection1   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 1783103 INFO  (OverseerStateUpdate-94905938134827012-127.0.0.1:54698_da_gh-n_0000000000) [n:127.0.0.1:54698_da_gh    ] o.a.s.c.o.ReplicaMutator Collection already exists with numShards=2
   [junit4]   2> 1783103 INFO  (OverseerStateUpdate-94905938134827012-127.0.0.1:54698_da_gh-n_0000000000) [n:127.0.0.1:54698_da_gh    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard2
   [junit4]   2> 1783206 INFO  (OverseerStateUpdate-94905938134827012-127.0.0.1:54698_da_gh-n_0000000000) [n:127.0.0.1:54698_da_gh    ] o.a.s.c.o.ZkStateWriter going to update_collection /collections/collection1/state.json version: 0
   [junit4]   2> 1783206 INFO  (zkCallback-1106-thread-1-processing-n:127.0.0.1:39166_da_gh) [n:127.0.0.1:39166_da_gh    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json for collection collection1 has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1783207 INFO  (zkCallback-1106-thread-1-processing-n:127.0.0.1:39166_da_gh) [n:127.0.0.1:39166_da_gh    ] o.a.s.c.c.ZkStateReader Updating data for collection1 from 0 to 1 
   [junit4]   2> 1784103 INFO  (coreLoadExecutor-3529-thread-1-processing-n:127.0.0.1:39166_da_gh) [n:127.0.0.1:39166_da_gh c:collection1   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1784103 INFO  (coreLoadExecutor-3529-thread-1-processing-n:127.0.0.1:39166_da_gh) [n:127.0.0.1:39166_da_gh c:collection1   x:collection1] o.a.s.c.ZkController Check for collection zkNode:collection1
   [junit4]   2> 1784104 INFO  (coreLoadExecutor-3529-thread-1-processing-n:127.0.0.1:39166_da_gh) [n:127.0.0.1:39166_da_gh c:collection1   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 1784104 INFO  (coreLoadExecutor-3529-thread-1-processing-n:127.0.0.1:39166_da_gh) [n:127.0.0.1:39166_da_gh c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/collection1
   [junit4]   2> 1784105 INFO  (coreLoadExecutor-3529-thread-1-processing-n:127.0.0.1:39166_da_gh) [n:127.0.0.1:39166_da_gh c:collection1   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1784105 INFO  (coreLoadExecutor-3529-thread-1-processing-n:127.0.0.1:39166_da_gh) [n:127.0.0.1:39166_da_gh c:collection1   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_99E29040E82FC4E0-001/shard-1-001/cores/collection1'
   [junit4]   2> 1784105 INFO  (coreLoadExecutor-3529-thread-1-processing-n:127.0.0.1:39166_da_gh) [n:127.0.0.1:39166_da_gh c:collection1   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1784105 INFO  (coreLoadExecutor-3529-thread-1-processing-n:127.0.0.1:39166_da_gh) [n:127.0.0.1:39166_da_gh c:collection1   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1784108 INFO  (coreLoadExecutor-3529-thread-1-processing-n:127.0.0.1:39166_da_gh) [n:127.0.0.1:39166_da_gh c:collection1   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 1784118 INFO  (coreLoadExecutor-3529-thread-1-processing-n:127.0.0.1:39166_da_gh) [n:127.0.0.1:39166_da_gh c:collection1   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 1784147 IN

[...truncated too long message...]

ctionContextBase.cancelElection(ElectionContext.java:167)
   [junit4]   2> 	at org.apache.solr.cloud.ShardLeaderElectionContext.cancelElection(ElectionContext.java:269)
   [junit4]   2> 	at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:441)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.access$200(LeaderElector.java:58)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector$ElectionWatcher.process(LeaderElector.java:388)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$3$1.run(SolrZkClient.java:264)
   [junit4]   2> 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
   [junit4]   2> 	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
   [junit4]   2> 	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 3210619 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[99E29040E82FC4E0]) [    ] o.e.j.s.ServerConnector Stopped ServerConnector@4e4ac73a{HTTP/1.1}{127.0.0.1:50191}
   [junit4]   2> 3210619 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[99E29040E82FC4E0]) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@1867ea5a{/,null,UNAVAILABLE}
   [junit4]   2> 3210621 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[99E29040E82FC4E0]) [n:127.0.0.1:50191_ c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.c.ZkTestServer connecting to 127.0.0.1:33316 33316
   [junit4]   2> 3210920 INFO  (Thread-167371) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:33316 33316
   [junit4]   2> 3210921 WARN  (Thread-167371) [    ] o.a.s.c.ZkTestServer Watch limit violations: 
   [junit4]   2> Maximum concurrent create/delete watches above limit:
   [junit4]   2> 
   [junit4]   2> 	11	/solr/aliases.json
   [junit4]   2> 	10	/solr/security.json
   [junit4]   2> 	10	/solr/configs/conf1
   [junit4]   2> 
   [junit4]   2> Maximum concurrent data watches above limit:
   [junit4]   2> 
   [junit4]   2> 	19	/solr/collections/all_in_lir/state.json
   [junit4]   2> 	11	/solr/clusterstate.json
   [junit4]   2> 	9	/solr/collections/collection1/state.json
   [junit4]   2> 	5	/solr/overseer_elect/election/94906005542666244-127.0.0.1:52876_-n_0000000000
   [junit4]   2> 	2	/solr/collections/all_in_lir/leader_elect/shard1/election/94906005542666252-core_node2-n_0000000000
   [junit4]   2> 	2	/solr/collections/all_in_lir/leader_elect/shard1/election/94906005542666267-core_node3-n_0000000051
   [junit4]   2> 
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2> 
   [junit4]   2> 	12	/solr/live_nodes
   [junit4]   2> 	11	/solr/collections
   [junit4]   2> 	3	/solr/overseer/collection-queue-work
   [junit4]   2> 	3	/solr/overseer/queue
   [junit4]   2> 
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=LeaderInitiatedRecoveryOnShardRestartTest -Dtests.method=testRestartWithAllInLIR -Dtests.seed=99E29040E82FC4E0 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=hu -Dtests.timezone=Africa/Dakar -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] FAILURE  403s J2 | LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR <<<
   [junit4]    > Throwable #1: java.lang.AssertionError: There are still nodes recoverying - waited for 330 seconds
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([99E29040E82FC4E0:4768FE3DD38DA613]:0)
   [junit4]    > 	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:170)
   [junit4]    > 	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:131)
   [junit4]    > 	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:126)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.waitForRecoveriesToFinish(AbstractFullDistribZkTestBase.java:830)
   [junit4]    > 	at org.apache.solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR(LeaderInitiatedRecoveryOnShardRestartTest.java:109)
   [junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:963)
   [junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:938)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 3210925 INFO  (SUITE-LeaderInitiatedRecoveryOnShardRestartTest-seed#[99E29040E82FC4E0]-worker) [n:127.0.0.1:50191_ c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_99E29040E82FC4E0-001
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene54): {}, docValues:{}, sim=RandomSimilarityProvider(queryNorm=true,coord=crazy): {}, locale=hu, timezone=Africa/Dakar
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_80 (64-bit)/cpus=4,threads=1,free=262252936,total=493355008
   [junit4]   2> NOTE: All tests run in this JVM: [TestCoreDiscovery, SaslZkACLProviderTest, ClusterStateTest, TestManagedSchemaFieldTypeResource, TestAuthenticationFramework, TestBlobHandler, AsyncCallRequestStatusResponseTest, SampleTest, TestLeaderInitiatedRecoveryThread, BufferStoreTest, ForceLeaderTest, TestSweetSpotSimilarityFactory, TestFileDictionaryLookup, DocValuesMultiTest, AtomicUpdatesTest, TestFastWriter, SuggesterTSTTest, TestBlendedInfixSuggestions, DebugComponentTest, AddBlockUpdateTest, HdfsBasicDistributedZkTest, CircularListTest, TestRuleBasedAuthorizationPlugin, AnalyticsMergeStrategyTest, CloneFieldUpdateProcessorFactoryTest, FacetPivotSmallTest, HardAutoCommitTest, TestElisionMultitermQuery, HdfsDirectoryFactoryTest, TestWriterPerf, TestSimpleQParserPlugin, TestSolrQueryParserResource, TestSort, NoCacheHeaderTest, PrimitiveFieldTypeTest, TestStressReorder, TestLRUStatsCache, SolrCoreCheckLockOnStartupTest, RequestLoggingTest, TestManagedResourceStorage, OverseerTaskQueueTest, TestPerFieldSimilarity, TestStandardQParsers, QueryParsingTest, TestSchemaVersionResource, TestSolrDeletionPolicy1, BigEndianAscendingWordSerializerTest, TestExtendedDismaxParser, AutoCommitTest, TestJsonFacets, TestIndexingPerformance, TestSolrIndexConfig, RankQueryTest, TestXIncludeConfig, TestRestManager, TestJmxMonitoredMap, EnumFieldTest, TestJettySolrRunner, StatsComponentTest, TestConfigOverlay, TestDynamicFieldResource, TestCustomDocTransformer, TestFieldTypeResource, TestRecovery, FullSolrCloudDistribCmdsTest, TestReplicationHandlerBackup, ShardSplitTest, TestReloadDeadlock, BinaryUpdateRequestHandlerTest, TestHashQParserPlugin, TestIndexSearcher, UpdateParamsTest, DocValuesMissingTest, TestSolrCloudWithKerberosAlt, SecurityConfHandlerTest, TestNamedUpdateProcessors, RegexBoostProcessorTest, DistributedFacetPivotLongTailTest, TestEmbeddedSolrServerConstructors, RecoveryAfterSoftCommitTest, TestDownShardTolerantSearch, BasicAuthIntegrationTest, DocExpirationUpdateProcessorFactoryTest, TestCSVResponseWriter, TestSolrQueryParser, PluginInfoTest, ShowFileRequestHandlerTest, TestSuggestSpellingConverter, ShardRoutingTest, RecoveryZkTest, LeaderElectionTest, TestZkChroot, TestDistributedSearch, SpellingQueryConverterTest, TestSolrJ, TestUtils, SliceStateTest, UUIDFieldTest, DistributedMLTComponentTest, TestHighlightDedupGrouping, TestSimpleTrackingShardHandler, CollectionStateFormat2Test, CollectionTooManyReplicasTest, DeleteLastCustomShardedReplicaTest, DeleteReplicaTest, DistribCursorPagingTest, LeaderInitiatedRecoveryOnShardRestartTest]
   [junit4] Completed [435/548] on J2 in 402.80s, 1 test, 1 failure <<< FAILURES!

[...truncated 374 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 548 suites (7 ignored), 2165 tests, 1 error, 1 failure, 32 ignored (18 assumptions) [seed: 99E29040E82FC4E0]

Total time: 199 minutes 25 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1023 - Still Failing

[Apache Jenkins Server <je...@builds.apache.org>]

Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1023/

13 tests failed.
FAILED:  org.apache.solr.cloud.BasicDistributedZkTest.test

Error Message:
commitWithin did not work on node: http://127.0.0.1:55429/_u/a/collection1 expected:<68> but was:<67>

Stack Trace:
java.lang.AssertionError: commitWithin did not work on node: http://127.0.0.1:55429/_u/a/collection1 expected:<68> but was:<67>
	at __randomizedtesting.SeedInfo.seed([21BCE81C1D8DC87E:A9E8D7C6B371A586]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.failNotEquals(Assert.java:647)
	at org.junit.Assert.assertEquals(Assert.java:128)
	at org.junit.Assert.assertEquals(Assert.java:472)
	at org.apache.solr.cloud.BasicDistributedZkTest.test(BasicDistributedZkTest.java:333)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:963)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:938)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Test abandoned because suite timeout was reached.

Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
	at __randomizedtesting.SeedInfo.seed([21BCE81C1D8DC87E]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.CollectionsAPIDistributedZkTest

Error Message:
Suite timeout exceeded (>= 7200000 msec).

Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
	at __randomizedtesting.SeedInfo.seed([21BCE81C1D8DC87E]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.CollectionsAPIDistributedZkTest

Error Message:
Captured an uncaught exception in thread: Thread[id=82049, name=OverseerThreadFactory-2368-thread-1, state=RUNNABLE, group=Overseer collection creation process.]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=82049, name=OverseerThreadFactory-2368-thread-1, state=RUNNABLE, group=Overseer collection creation process.]
Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.CollectionsAPIDistributedZkTest

Error Message:
Captured an uncaught exception in thread: Thread[id=82082, name=OverseerThreadFactory-2368-thread-2, state=RUNNABLE, group=Overseer collection creation process.]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=82082, name=OverseerThreadFactory-2368-thread-2, state=RUNNABLE, group=Overseer collection creation process.]
Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded
	at __randomizedtesting.SeedInfo.seed([21BCE81C1D8DC87E]:0)
	at org.apache.solr.cloud.OverseerCollectionMessageHandler.processResponse(OverseerCollectionMessageHandler.java:2395)
	at org.apache.solr.cloud.OverseerCollectionMessageHandler.processResponse(OverseerCollectionMessageHandler.java:2389)
	at org.apache.solr.cloud.OverseerCollectionMessageHandler.processResponses(OverseerCollectionMessageHandler.java:2282)
	at org.apache.solr.cloud.OverseerCollectionMessageHandler.createCollection(OverseerCollectionMessageHandler.java:2104)
	at org.apache.solr.cloud.OverseerCollectionMessageHandler.processMessage(OverseerCollectionMessageHandler.java:221)
	at org.apache.solr.cloud.OverseerTaskProcessor$Runner.run(OverseerTaskProcessor.java:434)
	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.CollectionsAPIDistributedZkTest

Error Message:
Captured an uncaught exception in thread: Thread[id=82048, name=Scheduler-311063407, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=82048, name=Scheduler-311063407, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.CollectionsAPIDistributedZkTest

Error Message:
Captured an uncaught exception in thread: Thread[id=82217, name=OverseerThreadFactory-2368-thread-5, state=RUNNABLE, group=Overseer collection creation process.]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=82217, name=OverseerThreadFactory-2368-thread-5, state=RUNNABLE, group=Overseer collection creation process.]
Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.CollectionsAPIDistributedZkTest

Error Message:
Captured an uncaught exception in thread: Thread[id=82476, name=collection4, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=82476, name=collection4, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: java.lang.RuntimeException: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:53143, http://127.0.0.1:56163, http://127.0.0.1:58347, http://127.0.0.1:51284, http://127.0.0.1:48252]
	at __randomizedtesting.SeedInfo.seed([21BCE81C1D8DC87E]:0)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:886)
Caused by: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:53143, http://127.0.0.1:56163, http://127.0.0.1:58347, http://127.0.0.1:51284, http://127.0.0.1:48252]
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:352)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1573)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1594)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:876)
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:53143: KeeperErrorCode = Session expired for /overseer/collection-queue-work/qnr-
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	... 7 more


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.CollectionsAPIDistributedZkTest

Error Message:
Captured an uncaught exception in thread: Thread[id=82474, name=collection2, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=82474, name=collection2, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: java.lang.RuntimeException: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:53143, http://127.0.0.1:56163, http://127.0.0.1:58347, http://127.0.0.1:51284, http://127.0.0.1:48252]
	at __randomizedtesting.SeedInfo.seed([21BCE81C1D8DC87E]:0)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:886)
Caused by: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:53143, http://127.0.0.1:56163, http://127.0.0.1:58347, http://127.0.0.1:51284, http://127.0.0.1:48252]
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:352)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1573)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1594)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:876)
Caused by: org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:53143
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:585)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	... 7 more
Caused by: java.net.SocketTimeoutException: Read timed out
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.read(SocketInputStream.java:152)
	at java.net.SocketInputStream.read(SocketInputStream.java:122)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:160)
	at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:84)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:273)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
	at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
	at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283)
	at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251)
	at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197)
	at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)
	at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)
	at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685)
	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487)
	at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:480)
	... 11 more


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.CollectionsAPIDistributedZkTest

Error Message:
Captured an uncaught exception in thread: Thread[id=82473, name=collection1, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=82473, name=collection1, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: java.lang.RuntimeException: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:53143, http://127.0.0.1:56163, http://127.0.0.1:58347, http://127.0.0.1:51284, http://127.0.0.1:48252]
	at __randomizedtesting.SeedInfo.seed([21BCE81C1D8DC87E]:0)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:886)
Caused by: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:53143, http://127.0.0.1:56163, http://127.0.0.1:58347, http://127.0.0.1:51284, http://127.0.0.1:48252]
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:352)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1573)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1594)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:876)
Caused by: org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:53143
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:585)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	... 7 more
Caused by: java.net.SocketTimeoutException: Read timed out
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.read(SocketInputStream.java:152)
	at java.net.SocketInputStream.read(SocketInputStream.java:122)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:160)
	at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:84)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:273)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
	at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
	at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283)
	at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251)
	at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197)
	at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)
	at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)
	at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685)
	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487)
	at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:480)
	... 11 more


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.CollectionsAPIDistributedZkTest

Error Message:
Captured an uncaught exception in thread: Thread[id=82475, name=collection3, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=82475, name=collection3, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: java.lang.RuntimeException: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:53143, http://127.0.0.1:56163, http://127.0.0.1:58347, http://127.0.0.1:51284, http://127.0.0.1:48252]
	at __randomizedtesting.SeedInfo.seed([21BCE81C1D8DC87E]:0)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:886)
Caused by: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:53143, http://127.0.0.1:56163, http://127.0.0.1:58347, http://127.0.0.1:51284, http://127.0.0.1:48252]
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:352)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1573)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1594)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:876)
Caused by: org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:53143
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:585)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	... 7 more
Caused by: java.net.SocketTimeoutException: Read timed out
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.read(SocketInputStream.java:152)
	at java.net.SocketInputStream.read(SocketInputStream.java:122)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:160)
	at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:84)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:273)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
	at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
	at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283)
	at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251)
	at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197)
	at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)
	at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)
	at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685)
	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487)
	at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:480)
	... 11 more


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.CollectionsAPIDistributedZkTest

Error Message:
Captured an uncaught exception in thread: Thread[id=82472, name=collection0, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=82472, name=collection0, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: java.lang.RuntimeException: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:53143, http://127.0.0.1:56163, http://127.0.0.1:58347, http://127.0.0.1:48252]
	at __randomizedtesting.SeedInfo.seed([21BCE81C1D8DC87E]:0)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:886)
Caused by: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:53143, http://127.0.0.1:56163, http://127.0.0.1:58347, http://127.0.0.1:48252]
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:352)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:883)
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:53143: KeeperErrorCode = Session expired for /overseer/collection-queue-work/qnr-
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	... 5 more


FAILED:  org.apache.solr.cloud.hdfs.HdfsUnloadDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=231725, name=testExecutor-8483-thread-8, state=RUNNABLE, group=TGRP-HdfsUnloadDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=231725, name=testExecutor-8483-thread-8, state=RUNNABLE, group=TGRP-HdfsUnloadDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:32849/l: Error CREATEing SolrCore 'multiunload7': Unable to create core [multiunload7] Caused by: Direct buffer memory
	at __randomizedtesting.SeedInfo.seed([21BCE81C1D8DC87E]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.BasicDistributedZkTest$1.run(BasicDistributedZkTest.java:550)
	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 10766 lines...]
   [junit4] Suite: org.apache.solr.cloud.BasicDistributedZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.BasicDistributedZkTest_21BCE81C1D8DC87E-001/init-core-data-001
   [junit4]   2> 1058571 INFO  (SUITE-BasicDistributedZkTest-seed#[21BCE81C1D8DC87E]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /_u/a
   [junit4]   2> 1058575 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 1058587 INFO  (Thread-7732) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1058587 INFO  (Thread-7732) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 1058676 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.ZkTestServer start zk server on port:35702
   [junit4]   2> 1058676 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1058677 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1058680 INFO  (zkCallback-951-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@677b2773 name:ZooKeeperConnection Watcher:127.0.0.1:35702 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1058680 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1058681 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1058681 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 1058685 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1058685 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1058687 INFO  (zkCallback-952-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@f890479 name:ZooKeeperConnection Watcher:127.0.0.1:35702/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1058687 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1058687 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1058687 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 1058690 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 1058693 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 1058694 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 1058697 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 1058697 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 1058701 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 1058702 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 1058707 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1058707 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1058709 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 1058709 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 1058712 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 1058712 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 1058716 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 1058716 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 1058717 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 1058718 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 1058720 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1058720 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1058723 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1058723 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1058725 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 1058725 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 1058727 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 1058727 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 1059017 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.BasicDistributedZkTest_21BCE81C1D8DC87E-001/control-001/cores/collection1
   [junit4]   2> 1059018 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 1059020 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@26fb18ab{/_u/a,null,AVAILABLE}
   [junit4]   2> 1059022 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.e.j.s.ServerConnector Started ServerConnector@2f10968b{HTTP/1.1}{127.0.0.1:58806}
   [junit4]   2> 1059022 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.e.j.s.Server Started @1061961ms
   [junit4]   2> 1059022 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/_u/a, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.BasicDistributedZkTest_21BCE81C1D8DC87E-001/control-001/cores, hostPort=58806, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.BasicDistributedZkTest_21BCE81C1D8DC87E-001/tempDir-001/control/data}
   [junit4]   2> 1059022 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@3827de24
   [junit4]   2> 1059022 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.BasicDistributedZkTest_21BCE81C1D8DC87E-001/control-001'
   [junit4]   2> 1059022 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1059023 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1059023 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1059024 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1059027 INFO  (zkCallback-953-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1b7cd4fc name:ZooKeeperConnection Watcher:127.0.0.1:35702/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1059027 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1059028 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1059029 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1059029 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.BasicDistributedZkTest_21BCE81C1D8DC87E-001/control-001/solr.xml
   [junit4]   2> 1059056 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.BasicDistributedZkTest_21BCE81C1D8DC87E-001/control-001/cores
   [junit4]   2> 1059057 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.CoreContainer New CoreContainer 313840481
   [junit4]   2> 1059057 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.BasicDistributedZkTest_21BCE81C1D8DC87E-001/control-001]
   [junit4]   2> 1059057 WARN  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.BasicDistributedZkTest_21BCE81C1D8DC87E-001/control-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.BasicDistributedZkTest_21BCE81C1D8DC87E-001/control-001/lib
   [junit4]   2> 1059057 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1059059 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1059059 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1059059 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1059060 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:35702/solr
   [junit4]   2> 1059060 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 1059060 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1059064 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1059067 INFO  (zkCallback-955-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@3b261b3a name:ZooKeeperConnection Watcher:127.0.0.1:35702 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1059067 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1059067 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1059070 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [n:127.0.0.1:58806__u%2Fa    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1059071 INFO  (zkCallback-956-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1ff98886 name:ZooKeeperConnection Watcher:127.0.0.1:35702/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1059071 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [n:127.0.0.1:58806__u%2Fa    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1059072 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [n:127.0.0.1:58806__u%2Fa    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 1059074 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [n:127.0.0.1:58806__u%2Fa    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 1059078 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [n:127.0.0.1:58806__u%2Fa    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 1059082 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [n:127.0.0.1:58806__u%2Fa    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 1059085 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [n:127.0.0.1:58806__u%2Fa    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 1059088 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [n:127.0.0.1:58806__u%2Fa    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 1059092 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [n:127.0.0.1:58806__u%2Fa    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 1059094 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [n:127.0.0.1:58806__u%2Fa    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 1059097 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [n:127.0.0.1:58806__u%2Fa    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 1059099 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [n:127.0.0.1:58806__u%2Fa    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:58806__u%2Fa
   [junit4]   2> 1059100 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [n:127.0.0.1:58806__u%2Fa    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:58806__u%2Fa
   [junit4]   2> 1059103 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [n:127.0.0.1:58806__u%2Fa    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 1059105 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [n:127.0.0.1:58806__u%2Fa    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 1059106 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [n:127.0.0.1:58806__u%2Fa    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1059107 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [n:127.0.0.1:58806__u%2Fa    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94901234297077764-127.0.0.1:58806__u%2Fa-n_0000000000
   [junit4]   2> 1059107 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [n:127.0.0.1:58806__u%2Fa    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:58806__u%2Fa
   [junit4]   2> 1059107 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [n:127.0.0.1:58806__u%2Fa    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 1059110 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [n:127.0.0.1:58806__u%2Fa    ] o.a.s.c.Overseer Overseer (id=94901234297077764-127.0.0.1:58806__u%2Fa-n_0000000000) starting
   [junit4]   2> 1059112 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [n:127.0.0.1:58806__u%2Fa    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 1059118 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [n:127.0.0.1:58806__u%2Fa    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 1059118 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [n:127.0.0.1:58806__u%2Fa    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1059119 INFO  (OverseerStateUpdate-94901234297077764-127.0.0.1:58806__u%2Fa-n_0000000000) [n:127.0.0.1:58806__u%2Fa    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 1059119 INFO  (OverseerCollectionConfigSetProcessor-94901234297077764-127.0.0.1:58806__u%2Fa-n_0000000000) [n:127.0.0.1:58806__u%2Fa    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 1059163 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [n:127.0.0.1:58806__u%2Fa    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1059163 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [n:127.0.0.1:58806__u%2Fa    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1059169 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [n:127.0.0.1:58806__u%2Fa    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.BasicDistributedZkTest_21BCE81C1D8DC87E-001/control-001/cores
   [junit4]   2> 1059171 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [n:127.0.0.1:58806__u%2Fa    ] o.a.s.c.SolrCore Created CoreDescriptor: {dataDir=data/, config=solrconfig.xml, shard=, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.BasicDistributedZkTest_21BCE81C1D8DC87E-001/control-001/cores/collection1, collection=control_collection, coreNodeName=, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.BasicDistributedZkTest_21BCE81C1D8DC87E-001/control-001/cores/collection1/, schema=schema.xml, loadOnStartup=true, name=collection1, transient=false, configSetProperties=configsetprops.json}
   [junit4]   2> 1059173 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [n:127.0.0.1:58806__u%2Fa    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.BasicDistributedZkTest_21BCE81C1D8DC87E-001/control-001/cores/collection1/
   [junit4]   2> 1059175 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [n:127.0.0.1:58806__u%2Fa    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 1059182 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 1059182 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1059185 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 1059187 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [n:127.0.0.1:58806__u%2Fa    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 1059187 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [n:127.0.0.1:58806__u%2Fa    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1059187 INFO  (OverseerStateUpdate-94901234297077764-127.0.0.1:58806__u%2Fa-n_0000000000) [n:127.0.0.1:58806__u%2Fa    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "base_url":"http://127.0.0.1:58806/_u/a",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "roles":null,
   [junit4]   2>   "node_name":"127.0.0.1:58806__u%2Fa",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down"} current state version: 0
   [junit4]   2> 1059188 INFO  (OverseerStateUpdate-94901234297077764-127.0.0.1:58806__u%2Fa-n_0000000000) [n:127.0.0.1:58806__u%2Fa    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "base_url":"http://127.0.0.1:58806/_u/a",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "roles":null,
   [junit4]   2>   "node_name":"127.0.0.1:58806__u%2Fa",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down"}
   [junit4]   2> 1059189 INFO  (OverseerStateUpdate-94901234297077764-127.0.0.1:58806__u%2Fa-n_0000000000) [n:127.0.0.1:58806__u%2Fa    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 1059189 INFO  (OverseerStateUpdate-94901234297077764-127.0.0.1:58806__u%2Fa-n_0000000000) [n:127.0.0.1:58806__u%2Fa    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 1059192 INFO  (zkCallback-956-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1060186 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1060186 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 1060187 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 1060187 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 1060193 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1060193 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.BasicDistributedZkTest_21BCE81C1D8DC87E-001/control-001/cores/collection1'
   [junit4]   2> 1060193 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1060193 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1060195 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 1060203 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 1060218 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 1060256 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1060263 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1060526 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 1060535 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 1060537 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 1060549 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1060553 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1060557 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1060558 WARN  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 1060558 WARN  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 1060558 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1060560 WARN  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 1060560 WARN  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 1060561 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 1060561 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 1060562 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 1060562 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.BasicDistributedZkTest_21BCE81C1D8DC87E-001/control-001/cores/collection1], dataDir=[null]
   [junit4]   2> 1060562 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@346bc66
   [junit4]   2> 1060563 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.BasicDistributedZkTest_21BCE81C1D8DC87E-001/control-001/cores/collection1/data
   [junit4]   2> 1060563 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.BasicDistributedZkTest_21BCE81C1D8DC87E-001/control-001/cores/collection1/data/index/
   [junit4]   2> 1060563 WARN  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.BasicDistributedZkTest_21BCE81C1D8DC87E-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 1060563 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.BasicDistributedZkTest_21BCE81C1D8DC87E-001/control-001/cores/collection1/data/index
   [junit4]   2> 1060564 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=10, maxMergeAtOnceExplicit=28, maxMergedSegmentMB=0.919921875, floorSegmentMB=0.5556640625, forceMergeDeletesPctAllowed=23.150084970163487, segmentsPerTier=15.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 1060564 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@5ac61053 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@159e7b4),segFN=segments_1,generation=1}
   [junit4]   2> 1060564 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 1060564 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.BasicDistributedZkTest_21BCE81C1D8DC87E-001/control-001/cores/collection1/data/
   [junit4]   2> 1060565 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.BasicDistributedZkTest_21BCE81C1D8DC87E-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 1060572 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 1060572 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 1060572 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 1060572 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1060572 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1060572 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 1060572 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1060573 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1060573 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 1060573 WARN  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
   [junit4]   2> 1060576 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1060577 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1060578 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1060580 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1060585 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/mbeans,/admin/luke,/admin/ping,/get,/update,/admin/plugins,/schema,/admin/logging,/admin/properties,/admin/file,/update/json,/replication,/update/csv,/update/json/docs,/config,/admin/system,/dump,standard,/admin/threads,/admin/segments
   [junit4]   2> 1060586 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 1060586 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1060586 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1060588 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 1060588 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 1060588 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=6, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=0.3857421875, noCFSRatio=0.16249597702253232]
   [junit4]   2> 1060589 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@5ac61053 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@159e7b4),segFN=segments_1,generation=1}
   [junit4]   2> 1060589 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 1060589 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@3a817a0e[collection1] main
   [junit4]   2> 1060589 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 1060590 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1060590 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 1060590 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 1060590 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 1060590 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1060591 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 1060591 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1060591 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 1060591 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1060591 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 1060591 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 1060591 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 1060592 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1518419750852493312
   [junit4]   2> 1060594 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 2.0ms to seed version buckets with highest version 1518419750852493312
   [junit4]   2> 1060594 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 1060594 INFO  (coreLoadExecutor-2966-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 1060594 INFO  (coreZkRegister-2960-thread-1-processing-x:collection1 c:control_collection s:shard1 n:127.0.0.1:58806__u%2Fa r:core_node1) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:58806/_u/a collection:control_collection shard:shard1
   [junit4]   2> 1060595 INFO  (searcherExecutor-2967-thread-1-processing-x:collection1 c:control_collection s:shard1 n:127.0.0.1:58806__u%2Fa r:core_node1) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@3a817a0e[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1060595 INFO  (coreZkRegister-2960-thread-1-processing-x:collection1 c:control_collection s:shard1 n:127.0.0.1:58806__u%2Fa r:core_node1) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 1060600 INFO  (coreZkRegister-2960-thread-1-processing-x:collection1 c:control_collection s:shard1 n:127.0.0.1:58806__u%2Fa r:core_node1) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94901234297077764-core_node1-n_0000000000
   [junit4]   2> 1060601 INFO  (coreZkRegister-2960-thread-1-processing-x:collection1 c:control_collection s:shard1 n:127.0.0.1:58806__u%2Fa r:core_node1) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 1060603 INFO  (OverseerStateUpdate-94901234297077764-127.0.0.1:58806__u%2Fa-n_0000000000) [n:127.0.0.1:58806__u%2Fa    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 1060606 INFO  (coreZkRegister-2960-thread-1-processing-x:collection1 c:control_collection s:shard1 n:127.0.0.1:58806__u%2Fa r:core_node1) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1060606 INFO  (coreZkRegister-2960-thread-1-processing-x:collection1 c:control_collection s:shard1 n:127.0.0.1:58806__u%2Fa r:core_node1) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1060606 INFO  (coreZkRegister-2960-thread-1-processing-x:collection1 c:control_collection s:shard1 n:127.0.0.1:58806__u%2Fa r:core_node1) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:58806/_u/a/collection1/
   [junit4]   2> 1060606 INFO  (coreZkRegister-2960-thread-1-processing-x:collection1 c:control_collection s:shard1 n:127.0.0.1:58806__u%2Fa r:core_node1) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 1060606 INFO  (coreZkRegister-2960-thread-1-processing-x:collection1 c:control_collection s:shard1 n:127.0.0.1:58806__u%2Fa r:core_node1) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:58806/_u/a/collection1/ has no replicas
   [junit4]   2> 1060607 INFO  (coreZkRegister-2960-thread-1-processing-x:collection1 c:control_collection s:shard1 n:127.0.0.1:58806__u%2Fa r:core_node1) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 1060611 INFO  (coreZkRegister-2960-thread-1-processing-x:collection1 c:control_collection s:shard1 n:127.0.0.1:58806__u%2Fa r:core_node1) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 1060613 INFO  (coreZkRegister-2960-thread-1-processing-x:collection1 c:control_collection s:shard1 n:127.0.0.1:58806__u%2Fa r:core_node1) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:58806/_u/a/collection1/ shard1
   [junit4]   2> 1060614 INFO  (OverseerStateUpdate-94901234297077764-127.0.0.1:58806__u%2Fa-n_0000000000) [n:127.0.0.1:58806__u%2Fa    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:58806/_u/a",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 1
   [junit4]   2> 1060693 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1060695 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1060698 INFO  (zkCallback-958-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@16260ac3 name:ZooKeeperConnection Watcher:127.0.0.1:35702/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1060698 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1060698 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1060698 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1060702 INFO  (TEST-BasicDistributedZkTest.test-seed#[21BCE81C1D8DC87E]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 1060717 INFO  (zkCallback-956-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1060717 INFO  (zkCallback-958-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1060765 INFO  (coreZkRegister-2960-thread-1-processing-x:collection1 c:control_collection s:shard1 n:127.0.0.1:58806__u%2Fa r:core_node1) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:58806/_u/a/collection1/ and leader is http://127.0.0.1:58806/_u/a/collection1/
   [junit4]   2> 1060765 INFO  (coreZkRegister-2960-thread-1-processing-x:collection1 c:control_collection s:shard1 n:127.0.0.1:58806__u%2Fa r:core_node1) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:58806/_u/a
   [junit4]   2> 1060765 INFO  (coreZkRegister-2960-thread-1-processing-x:collection1 c:control_collection s:shard1 n:127.0.0.1:58806__u%2Fa r:core_node1) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 1060765 INFO  (coreZkRegister-2960-thread-1-processing-x:collection1 c:control_collection s:shard1 n:127.0.0.1:58806__u%2Fa r:core_node1) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 1060765 INFO  (coreZkRegister-2960-thread-1-processing-x:collection1 c:control_collection s:shard1 n:127.0.0.1:58806__u%2Fa r:core_node1) [n:127.0.0.1:58806__u%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1060767 INFO  (OverseerStateUpdate-94901234297077764-127.0.0.1:58806__u%2Fa-n_0000000000) [n:127.0.0.1:58806__u%2Fa    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "base_url":"http://127.0.0.1:58806/_u/a",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "node_name":"127.0.0.1:58806__u%2Fa",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "state":"active"} current state version: 2
   [junit4]   2> 1060767 INFO  (OverseerStateUpdate-94901234297077764-127.0.0.1:58806__u%2Fa-n_0000000000) [n:127.0.0.1:58806__u%2Fa    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "base_url":"http://127.0.0.1:58806/_u/a",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "node_name":"127.0.0.1:58806__u%2Fa",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 1060870 INFO  (zkCallback-958-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1060870 INFO  (zkCallback-956-thread-1-processing-n:127.0.0.1:58806__u%2Fa) [n:127.0.0.1:58806__u%2Fa    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/cluste

[...truncated too long message...]

rg.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:480)
   [junit4]    > 	... 11 moreThrowable #9: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=82475, name=collection3, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    > Caused by: java.lang.RuntimeException: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:53143, http://127.0.0.1:56163, http://127.0.0.1:58347, http://127.0.0.1:51284, http://127.0.0.1:48252]
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([21BCE81C1D8DC87E]:0)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:886)
   [junit4]    > Caused by: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:53143, http://127.0.0.1:56163, http://127.0.0.1:58347, http://127.0.0.1:51284, http://127.0.0.1:48252]
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:352)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1573)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1594)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:876)
   [junit4]    > Caused by: org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:53143
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:585)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
   [junit4]    > 	... 7 more
   [junit4]    > Caused by: java.net.SocketTimeoutException: Read timed out
   [junit4]    > 	at java.net.SocketInputStream.socketRead0(Native Method)
   [junit4]    > 	at java.net.SocketInputStream.read(SocketInputStream.java:152)
   [junit4]    > 	at java.net.SocketInputStream.read(SocketInputStream.java:122)
   [junit4]    > 	at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:160)
   [junit4]    > 	at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:84)
   [junit4]    > 	at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:273)
   [junit4]    > 	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
   [junit4]    > 	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
   [junit4]    > 	at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
   [junit4]    > 	at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283)
   [junit4]    > 	at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251)
   [junit4]    > 	at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197)
   [junit4]    > 	at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)
   [junit4]    > 	at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)
   [junit4]    > 	at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685)
   [junit4]    > 	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487)
   [junit4]    > 	at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
   [junit4]    > 	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
   [junit4]    > 	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
   [junit4]    > 	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:480)
   [junit4]    > 	... 11 moreThrowable #10: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=82472, name=collection0, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    > Caused by: java.lang.RuntimeException: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:53143, http://127.0.0.1:56163, http://127.0.0.1:58347, http://127.0.0.1:48252]
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([21BCE81C1D8DC87E]:0)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:886)
   [junit4]    > Caused by: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:53143, http://127.0.0.1:56163, http://127.0.0.1:58347, http://127.0.0.1:48252]
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:352)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:883)
   [junit4]    > Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:53143: KeeperErrorCode = Session expired for /overseer/collection-queue-work/qnr-
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
   [junit4]    > 	... 5 more
   [junit4] Completed [506/548] on J1 in 7229.31s, 1 test, 11 errors <<< FAILURES!

[...truncated 129 lines...]
   [junit4] JVM J1: stdout was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J1-20151121_033414_270.sysout
   [junit4] >>> JVM J1: stdout (verbatim) ----
   [junit4] java.lang.OutOfMemoryError: GC overhead limit exceeded
   [junit4] Dumping heap to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/heapdumps/java_pid11232.hprof ...
   [junit4] Heap dump file created [642691216 bytes in 15.265 secs]
   [junit4] <<< JVM J1: EOF ----

   [junit4] JVM J1: stderr was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J1-20151121_033414_270.syserr
   [junit4] >>> JVM J1: stderr (verbatim) ----
   [junit4] 
   [junit4] Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "OverseerThreadFactory-2368-thread-1"
   [junit4] <<< JVM J1: EOF ----

[...truncated 13 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 548 suites (7 ignored), 2165 tests, 10 suite-level errors, 2 errors, 1 failure, 159 ignored (18 assumptions) [seed: 21BCE81C1D8DC87E]

Total time: 278 minutes 25 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1022 - Still Failing

[Apache Jenkins Server <je...@builds.apache.org>]

Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1022/

2 tests failed.
FAILED:  org.apache.solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR

Error Message:
There are still nodes recoverying - waited for 330 seconds

Stack Trace:
java.lang.AssertionError: There are still nodes recoverying - waited for 330 seconds
	at __randomizedtesting.SeedInfo.seed([A3790642C165612C:7DF3683FFAC703DF]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:170)
	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:131)
	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:126)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.waitForRecoveriesToFinish(AbstractFullDistribZkTestBase.java:830)
	at org.apache.solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR(LeaderInitiatedRecoveryOnShardRestartTest.java:109)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:963)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:938)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.handler.TestReplicationHandler.doTestReplicateAfterCoreReload

Error Message:
expected:<[{indexVersion=1447979710547,generation=2,filelist=[_11lq.cfe, _11lq.cfs, _11lq.si, _11xj.fdt, _11xj.fdx, _11xj.fnm, _11xj.nvd, _11xj.nvm, _11xj.si, _11xj_FSTOrd50_0.doc, _11xj_FSTOrd50_0.tbk, _11xj_FSTOrd50_0.tix, _11xl.cfe, _11xl.cfs, _11xl.si, _11xq.cfe, _11xq.cfs, _11xq.si, _11xs.fdt, _11xs.fdx, _11xs.fnm, _11xs.nvd, _11xs.nvm, _11xs.si, _11xs_FSTOrd50_0.doc, _11xs_FSTOrd50_0.tbk, _11xs_FSTOrd50_0.tix, _11xu.fdt, _11xu.fdx, _11xu.fnm, _11xu.nvd, _11xu.nvm, _11xu.si, _11xu_FSTOrd50_0.doc, _11xu_FSTOrd50_0.tbk, _11xu_FSTOrd50_0.tix, _11xv.fdt, _11xv.fdx, _11xv.fnm, _11xv.nvd, _11xv.nvm, _11xv.si, _11xv_FSTOrd50_0.doc, _11xv_FSTOrd50_0.tbk, _11xv_FSTOrd50_0.tix, _11xw.fdt, _11xw.fdx, _11xw.fnm, _11xw.nvd, _11xw.nvm, _11xw.si, _11xw_FSTOrd50_0.doc, _11xw_FSTOrd50_0.tbk, _11xw_FSTOrd50_0.tix, _11xz.fdt, _11xz.fdx, _11xz.fnm, _11xz.nvd, _11xz.nvm, _11xz.si, _11xz_FSTOrd50_0.doc, _11xz_FSTOrd50_0.tbk, _11xz_FSTOrd50_0.tix, _11y4.fdt, _11y4.fdx, _11y4.fnm, _11y4.nvd, _11y4.nvm, _11y4.si, _11y4_FSTOrd50_0.doc, _11y4_FSTOrd50_0.tbk, _11y4_FSTOrd50_0.tix, _11y6.fdt, _11y6.fdx, _11y6.fnm, _11y6.nvd, _11y6.nvm, _11y6.si, _11y6_FSTOrd50_0.doc, _11y6_FSTOrd50_0.tbk, _11y6_FSTOrd50_0.tix, _11y7.cfe, _11y7.cfs, _11y7.si, _11y8.fdt, _11y8.fdx, _11y8.fnm, _11y8.nvd, _11y8.nvm, _11y8.si, _11y8_FSTOrd50_0.doc, _11y8_FSTOrd50_0.tbk, _11y8_FSTOrd50_0.tix, _11ya.fdt, _11ya.fdx, _11ya.fnm, _11ya.nvd, _11ya.nvm, _11ya.si, _11ya_FSTOrd50_0.doc, _11ya_FSTOrd50_0.tbk, _11ya_FSTOrd50_0.tix, _11yc.fdt, _11yc.fdx, _11yc.fnm, _11yc.nvd, _11yc.nvm, _11yc.si, _11yc_FSTOrd50_0.doc, _11yc_FSTOrd50_0.tbk, _11yc_FSTOrd50_0.tix, _11yd.fdt, _11yd.fdx, _11yd.fnm, _11yd.nvd, _11yd.nvm, _11yd.si, _11yd_FSTOrd50_0.doc, _11yd_FSTOrd50_0.tbk, _11yd_FSTOrd50_0.tix, _11ye.fdt, _11ye.fdx, _11ye.fnm, _11ye.nvd, _11ye.nvm, _11ye.si, _11ye_FSTOrd50_0.doc, _11ye_FSTOrd50_0.tbk, _11ye_FSTOrd50_0.tix, _126b.fdt, _126b.fdx, _126b.fnm, _126b.nvd, _126b.nvm, _126b.si, _126b_FSTOrd50_0.doc, _126b_FSTOrd50_0.tbk, _126b_FSTOrd50_0.tix, _126g.fdt, _126g.fdx, _126g.fnm, _126g.nvd, _126g.nvm, _126g.si, _126g_FSTOrd50_0.doc, _126g_FSTOrd50_0.tbk, _126g_FSTOrd50_0.tix, _126h.cfe, _126h.cfs, _126h.si, _126p.fdt, _126p.fdx, _126p.fnm, _126p.nvd, _126p.nvm, _126p.si, _126p_FSTOrd50_0.doc, _126p_FSTOrd50_0.tbk, _126p_FSTOrd50_0.tix, _126v.fdt, _126v.fdx, _126v.fnm, _126v.nvd, _126v.nvm, _126v.si, _126v_FSTOrd50_0.doc, _126v_FSTOrd50_0.tbk, _126v_FSTOrd50_0.tix, _1274.fdt, _1274.fdx, _1274.fnm, _1274.nvd, _1274.nvm, _1274.si, _1274_FSTOrd50_0.doc, _1274_FSTOrd50_0.tbk, _1274_FSTOrd50_0.tix, _1277.fdt, _1277.fdx, _1277.fnm, _1277.nvd, _1277.nvm, _1277.si, _1277_FSTOrd50_0.doc, _1277_FSTOrd50_0.tbk, _1277_FSTOrd50_0.tix, _1278.fdt, _1278.fdx, _1278.fnm, _1278.nvd, _1278.nvm, _1278.si, _1278_FSTOrd50_0.doc, _1278_FSTOrd50_0.tbk, _1278_FSTOrd50_0.tix, _1279.cfe, _1279.cfs, _1279.si, _127a.fdt, _127a.fdx, _127a.fnm, _127a.nvd, _127a.nvm, _127a.si, _127a_FSTOrd50_0.doc, _127a_FSTOrd50_0.tbk, _127a_FSTOrd50_0.tix, _127b.fdt, _127b.fdx, _127b.fnm, _127b.nvd, _127b.nvm, _127b.si, _127b_FSTOrd50_0.doc, _127b_FSTOrd50_0.tbk, _127b_FSTOrd50_0.tix, _127c.fdt, _127c.fdx, _127c.fnm, _127c.nvd, _127c.nvm, _127c.si, _127c_FSTOrd50_0.doc, _127c_FSTOrd50_0.tbk, _127c_FSTOrd50_0.tix, _127d.fdt, _127d.fdx, _127d.fnm, _127d.nvd, _127d.nvm, _127d.si, _127d_FSTOrd50_0.doc, _127d_FSTOrd50_0.tbk, _127d_FSTOrd50_0.tix, _127e.fdt, _127e.fdx, _127e.fnm, _127e.nvd, _127e.nvm, _127e.si, _127e_FSTOrd50_0.doc, _127e_FSTOrd50_0.tbk, _127e_FSTOrd50_0.tix, _127f.fdt, _127f.fdx, _127f.fnm, _127f.nvd, _127f.nvm, _127f.si, _127f_FSTOrd50_0.doc, _127f_FSTOrd50_0.tbk, _127f_FSTOrd50_0.tix, _127h.fdt, _127h.fdx, _127h.fnm, _127h.nvd, _127h.nvm, _127h.si, _127h_FSTOrd50_0.doc, _127h_FSTOrd50_0.tbk, _127h_FSTOrd50_0.tix, segments_2]}]> but was:<[{indexVersion=1447979710547,generation=2,filelist=[_11lq.cfe, _11lq.cfs, _11lq.si, _11xj.fdt, _11xj.fdx, _11xj.fnm, _11xj.nvd, _11xj.nvm, _11xj.si, _11xj_FSTOrd50_0.doc, _11xj_FSTOrd50_0.tbk, _11xj_FSTOrd50_0.tix, _11xl.cfe, _11xl.cfs, _11xl.si, _11xq.cfe, _11xq.cfs, _11xq.si, _11xs.fdt, _11xs.fdx, _11xs.fnm, _11xs.nvd, _11xs.nvm, _11xs.si, _11xs_FSTOrd50_0.doc, _11xs_FSTOrd50_0.tbk, _11xs_FSTOrd50_0.tix, _11xu.fdt, _11xu.fdx, _11xu.fnm, _11xu.nvd, _11xu.nvm, _11xu.si, _11xu_FSTOrd50_0.doc, _11xu_FSTOrd50_0.tbk, _11xu_FSTOrd50_0.tix, _11xv.fdt, _11xv.fdx, _11xv.fnm, _11xv.nvd, _11xv.nvm, _11xv.si, _11xv_FSTOrd50_0.doc, _11xv_FSTOrd50_0.tbk, _11xv_FSTOrd50_0.tix, _11xw.fdt, _11xw.fdx, _11xw.fnm, _11xw.nvd, _11xw.nvm, _11xw.si, _11xw_FSTOrd50_0.doc, _11xw_FSTOrd50_0.tbk, _11xw_FSTOrd50_0.tix, _11xz.fdt, _11xz.fdx, _11xz.fnm, _11xz.nvd, _11xz.nvm, _11xz.si, _11xz_FSTOrd50_0.doc, _11xz_FSTOrd50_0.tbk, _11xz_FSTOrd50_0.tix, _11y4.fdt, _11y4.fdx, _11y4.fnm, _11y4.nvd, _11y4.nvm, _11y4.si, _11y4_FSTOrd50_0.doc, _11y4_FSTOrd50_0.tbk, _11y4_FSTOrd50_0.tix, _11y6.fdt, _11y6.fdx, _11y6.fnm, _11y6.nvd, _11y6.nvm, _11y6.si, _11y6_FSTOrd50_0.doc, _11y6_FSTOrd50_0.tbk, _11y6_FSTOrd50_0.tix, _11y7.cfe, _11y7.cfs, _11y7.si, _11y8.fdt, _11y8.fdx, _11y8.fnm, _11y8.nvd, _11y8.nvm, _11y8.si, _11y8_FSTOrd50_0.doc, _11y8_FSTOrd50_0.tbk, _11y8_FSTOrd50_0.tix, _11ya.fdt, _11ya.fdx, _11ya.fnm, _11ya.nvd, _11ya.nvm, _11ya.si, _11ya_FSTOrd50_0.doc, _11ya_FSTOrd50_0.tbk, _11ya_FSTOrd50_0.tix, _11yc.fdt, _11yc.fdx, _11yc.fnm, _11yc.nvd, _11yc.nvm, _11yc.si, _11yc_FSTOrd50_0.doc, _11yc_FSTOrd50_0.tbk, _11yc_FSTOrd50_0.tix, _11yd.fdt, _11yd.fdx, _11yd.fnm, _11yd.nvd, _11yd.nvm, _11yd.si, _11yd_FSTOrd50_0.doc, _11yd_FSTOrd50_0.tbk, _11yd_FSTOrd50_0.tix, _11ye.fdt, _11ye.fdx, _11ye.fnm, _11ye.nvd, _11ye.nvm, _11ye.si, _11ye_FSTOrd50_0.doc, _11ye_FSTOrd50_0.tbk, _11ye_FSTOrd50_0.tix, _126b.fdt, _126b.fdx, _126b.fnm, _126b.nvd, _126b.nvm, _126b.si, _126b_FSTOrd50_0.doc, _126b_FSTOrd50_0.tbk, _126b_FSTOrd50_0.tix, _126g.fdt, _126g.fdx, _126g.fnm, _126g.nvd, _126g.nvm, _126g.si, _126g_FSTOrd50_0.doc, _126g_FSTOrd50_0.tbk, _126g_FSTOrd50_0.tix, _126h.cfe, _126h.cfs, _126h.si, _126p.fdt, _126p.fdx, _126p.fnm, _126p.nvd, _126p.nvm, _126p.si, _126p_FSTOrd50_0.doc, _126p_FSTOrd50_0.tbk, _126p_FSTOrd50_0.tix, _126v.fdt, _126v.fdx, _126v.fnm, _126v.nvd, _126v.nvm, _126v.si, _126v_FSTOrd50_0.doc, _126v_FSTOrd50_0.tbk, _126v_FSTOrd50_0.tix, _1274.fdt, _1274.fdx, _1274.fnm, _1274.nvd, _1274.nvm, _1274.si, _1274_FSTOrd50_0.doc, _1274_FSTOrd50_0.tbk, _1274_FSTOrd50_0.tix, _1277.fdt, _1277.fdx, _1277.fnm, _1277.nvd, _1277.nvm, _1277.si, _1277_FSTOrd50_0.doc, _1277_FSTOrd50_0.tbk, _1277_FSTOrd50_0.tix, _1278.fdt, _1278.fdx, _1278.fnm, _1278.nvd, _1278.nvm, _1278.si, _1278_FSTOrd50_0.doc, _1278_FSTOrd50_0.tbk, _1278_FSTOrd50_0.tix, _1279.cfe, _1279.cfs, _1279.si, _127a.fdt, _127a.fdx, _127a.fnm, _127a.nvd, _127a.nvm, _127a.si, _127a_FSTOrd50_0.doc, _127a_FSTOrd50_0.tbk, _127a_FSTOrd50_0.tix, _127b.fdt, _127b.fdx, _127b.fnm, _127b.nvd, _127b.nvm, _127b.si, _127b_FSTOrd50_0.doc, _127b_FSTOrd50_0.tbk, _127b_FSTOrd50_0.tix, _127c.fdt, _127c.fdx, _127c.fnm, _127c.nvd, _127c.nvm, _127c.si, _127c_FSTOrd50_0.doc, _127c_FSTOrd50_0.tbk, _127c_FSTOrd50_0.tix, _127d.fdt, _127d.fdx, _127d.fnm, _127d.nvd, _127d.nvm, _127d.si, _127d_FSTOrd50_0.doc, _127d_FSTOrd50_0.tbk, _127d_FSTOrd50_0.tix, _127e.fdt, _127e.fdx, _127e.fnm, _127e.nvd, _127e.nvm, _127e.si, _127e_FSTOrd50_0.doc, _127e_FSTOrd50_0.tbk, _127e_FSTOrd50_0.tix, _127f.fdt, _127f.fdx, _127f.fnm, _127f.nvd, _127f.nvm, _127f.si, _127f_FSTOrd50_0.doc, _127f_FSTOrd50_0.tbk, _127f_FSTOrd50_0.tix, _127h.fdt, _127h.fdx, _127h.fnm, _127h.nvd, _127h.nvm, _127h.si, _127h_FSTOrd50_0.doc, _127h_FSTOrd50_0.tbk, _127h_FSTOrd50_0.tix, segments_2]}, {indexVersion=1447979710547,generation=3,filelist=[_11yg.cfe, _11yg.cfs, _11yg.si, _126b.fdt, _126b.fdx, _126b.fnm, _126b.nvd, _126b.nvm, _126b.si, _126b_FSTOrd50_0.doc, _126b_FSTOrd50_0.tbk, _126b_FSTOrd50_0.tix, _126g.fdt, _126g.fdx, _126g.fnm, _126g.nvd, _126g.nvm, _126g.si, _126g_FSTOrd50_0.doc, _126g_FSTOrd50_0.tbk, _126g_FSTOrd50_0.tix, _126p.fdt, _126p.fdx, _126p.fnm, _126p.nvd, _126p.nvm, _126p.si, _126p_FSTOrd50_0.doc, _126p_FSTOrd50_0.tbk, _126p_FSTOrd50_0.tix, _126v.fdt, _126v.fdx, _126v.fnm, _126v.nvd, _126v.nvm, _126v.si, _126v_FSTOrd50_0.doc, _126v_FSTOrd50_0.tbk, _126v_FSTOrd50_0.tix, _127f.fdt, _127f.fdx, _127f.fnm, _127f.nvd, _127f.nvm, _127f.si, _127f_FSTOrd50_0.doc, _127f_FSTOrd50_0.tbk, _127f_FSTOrd50_0.tix, _127g.cfe, _127g.cfs, _127g.si, _127h.fdt, _127h.fdx, _127h.fnm, _127h.nvd, _127h.nvm, _127h.si, _127h_FSTOrd50_0.doc, _127h_FSTOrd50_0.tbk, _127h_FSTOrd50_0.tix, segments_3]}]>

Stack Trace:
java.lang.AssertionError: expected:<[{indexVersion=1447979710547,generation=2,filelist=[_11lq.cfe, _11lq.cfs, _11lq.si, _11xj.fdt, _11xj.fdx, _11xj.fnm, _11xj.nvd, _11xj.nvm, _11xj.si, _11xj_FSTOrd50_0.doc, _11xj_FSTOrd50_0.tbk, _11xj_FSTOrd50_0.tix, _11xl.cfe, _11xl.cfs, _11xl.si, _11xq.cfe, _11xq.cfs, _11xq.si, _11xs.fdt, _11xs.fdx, _11xs.fnm, _11xs.nvd, _11xs.nvm, _11xs.si, _11xs_FSTOrd50_0.doc, _11xs_FSTOrd50_0.tbk, _11xs_FSTOrd50_0.tix, _11xu.fdt, _11xu.fdx, _11xu.fnm, _11xu.nvd, _11xu.nvm, _11xu.si, _11xu_FSTOrd50_0.doc, _11xu_FSTOrd50_0.tbk, _11xu_FSTOrd50_0.tix, _11xv.fdt, _11xv.fdx, _11xv.fnm, _11xv.nvd, _11xv.nvm, _11xv.si, _11xv_FSTOrd50_0.doc, _11xv_FSTOrd50_0.tbk, _11xv_FSTOrd50_0.tix, _11xw.fdt, _11xw.fdx, _11xw.fnm, _11xw.nvd, _11xw.nvm, _11xw.si, _11xw_FSTOrd50_0.doc, _11xw_FSTOrd50_0.tbk, _11xw_FSTOrd50_0.tix, _11xz.fdt, _11xz.fdx, _11xz.fnm, _11xz.nvd, _11xz.nvm, _11xz.si, _11xz_FSTOrd50_0.doc, _11xz_FSTOrd50_0.tbk, _11xz_FSTOrd50_0.tix, _11y4.fdt, _11y4.fdx, _11y4.fnm, _11y4.nvd, _11y4.nvm, _11y4.si, _11y4_FSTOrd50_0.doc, _11y4_FSTOrd50_0.tbk, _11y4_FSTOrd50_0.tix, _11y6.fdt, _11y6.fdx, _11y6.fnm, _11y6.nvd, _11y6.nvm, _11y6.si, _11y6_FSTOrd50_0.doc, _11y6_FSTOrd50_0.tbk, _11y6_FSTOrd50_0.tix, _11y7.cfe, _11y7.cfs, _11y7.si, _11y8.fdt, _11y8.fdx, _11y8.fnm, _11y8.nvd, _11y8.nvm, _11y8.si, _11y8_FSTOrd50_0.doc, _11y8_FSTOrd50_0.tbk, _11y8_FSTOrd50_0.tix, _11ya.fdt, _11ya.fdx, _11ya.fnm, _11ya.nvd, _11ya.nvm, _11ya.si, _11ya_FSTOrd50_0.doc, _11ya_FSTOrd50_0.tbk, _11ya_FSTOrd50_0.tix, _11yc.fdt, _11yc.fdx, _11yc.fnm, _11yc.nvd, _11yc.nvm, _11yc.si, _11yc_FSTOrd50_0.doc, _11yc_FSTOrd50_0.tbk, _11yc_FSTOrd50_0.tix, _11yd.fdt, _11yd.fdx, _11yd.fnm, _11yd.nvd, _11yd.nvm, _11yd.si, _11yd_FSTOrd50_0.doc, _11yd_FSTOrd50_0.tbk, _11yd_FSTOrd50_0.tix, _11ye.fdt, _11ye.fdx, _11ye.fnm, _11ye.nvd, _11ye.nvm, _11ye.si, _11ye_FSTOrd50_0.doc, _11ye_FSTOrd50_0.tbk, _11ye_FSTOrd50_0.tix, _126b.fdt, _126b.fdx, _126b.fnm, _126b.nvd, _126b.nvm, _126b.si, _126b_FSTOrd50_0.doc, _126b_FSTOrd50_0.tbk, _126b_FSTOrd50_0.tix, _126g.fdt, _126g.fdx, _126g.fnm, _126g.nvd, _126g.nvm, _126g.si, _126g_FSTOrd50_0.doc, _126g_FSTOrd50_0.tbk, _126g_FSTOrd50_0.tix, _126h.cfe, _126h.cfs, _126h.si, _126p.fdt, _126p.fdx, _126p.fnm, _126p.nvd, _126p.nvm, _126p.si, _126p_FSTOrd50_0.doc, _126p_FSTOrd50_0.tbk, _126p_FSTOrd50_0.tix, _126v.fdt, _126v.fdx, _126v.fnm, _126v.nvd, _126v.nvm, _126v.si, _126v_FSTOrd50_0.doc, _126v_FSTOrd50_0.tbk, _126v_FSTOrd50_0.tix, _1274.fdt, _1274.fdx, _1274.fnm, _1274.nvd, _1274.nvm, _1274.si, _1274_FSTOrd50_0.doc, _1274_FSTOrd50_0.tbk, _1274_FSTOrd50_0.tix, _1277.fdt, _1277.fdx, _1277.fnm, _1277.nvd, _1277.nvm, _1277.si, _1277_FSTOrd50_0.doc, _1277_FSTOrd50_0.tbk, _1277_FSTOrd50_0.tix, _1278.fdt, _1278.fdx, _1278.fnm, _1278.nvd, _1278.nvm, _1278.si, _1278_FSTOrd50_0.doc, _1278_FSTOrd50_0.tbk, _1278_FSTOrd50_0.tix, _1279.cfe, _1279.cfs, _1279.si, _127a.fdt, _127a.fdx, _127a.fnm, _127a.nvd, _127a.nvm, _127a.si, _127a_FSTOrd50_0.doc, _127a_FSTOrd50_0.tbk, _127a_FSTOrd50_0.tix, _127b.fdt, _127b.fdx, _127b.fnm, _127b.nvd, _127b.nvm, _127b.si, _127b_FSTOrd50_0.doc, _127b_FSTOrd50_0.tbk, _127b_FSTOrd50_0.tix, _127c.fdt, _127c.fdx, _127c.fnm, _127c.nvd, _127c.nvm, _127c.si, _127c_FSTOrd50_0.doc, _127c_FSTOrd50_0.tbk, _127c_FSTOrd50_0.tix, _127d.fdt, _127d.fdx, _127d.fnm, _127d.nvd, _127d.nvm, _127d.si, _127d_FSTOrd50_0.doc, _127d_FSTOrd50_0.tbk, _127d_FSTOrd50_0.tix, _127e.fdt, _127e.fdx, _127e.fnm, _127e.nvd, _127e.nvm, _127e.si, _127e_FSTOrd50_0.doc, _127e_FSTOrd50_0.tbk, _127e_FSTOrd50_0.tix, _127f.fdt, _127f.fdx, _127f.fnm, _127f.nvd, _127f.nvm, _127f.si, _127f_FSTOrd50_0.doc, _127f_FSTOrd50_0.tbk, _127f_FSTOrd50_0.tix, _127h.fdt, _127h.fdx, _127h.fnm, _127h.nvd, _127h.nvm, _127h.si, _127h_FSTOrd50_0.doc, _127h_FSTOrd50_0.tbk, _127h_FSTOrd50_0.tix, segments_2]}]> but was:<[{indexVersion=1447979710547,generation=2,filelist=[_11lq.cfe, _11lq.cfs, _11lq.si, _11xj.fdt, _11xj.fdx, _11xj.fnm, _11xj.nvd, _11xj.nvm, _11xj.si, _11xj_FSTOrd50_0.doc, _11xj_FSTOrd50_0.tbk, _11xj_FSTOrd50_0.tix, _11xl.cfe, _11xl.cfs, _11xl.si, _11xq.cfe, _11xq.cfs, _11xq.si, _11xs.fdt, _11xs.fdx, _11xs.fnm, _11xs.nvd, _11xs.nvm, _11xs.si, _11xs_FSTOrd50_0.doc, _11xs_FSTOrd50_0.tbk, _11xs_FSTOrd50_0.tix, _11xu.fdt, _11xu.fdx, _11xu.fnm, _11xu.nvd, _11xu.nvm, _11xu.si, _11xu_FSTOrd50_0.doc, _11xu_FSTOrd50_0.tbk, _11xu_FSTOrd50_0.tix, _11xv.fdt, _11xv.fdx, _11xv.fnm, _11xv.nvd, _11xv.nvm, _11xv.si, _11xv_FSTOrd50_0.doc, _11xv_FSTOrd50_0.tbk, _11xv_FSTOrd50_0.tix, _11xw.fdt, _11xw.fdx, _11xw.fnm, _11xw.nvd, _11xw.nvm, _11xw.si, _11xw_FSTOrd50_0.doc, _11xw_FSTOrd50_0.tbk, _11xw_FSTOrd50_0.tix, _11xz.fdt, _11xz.fdx, _11xz.fnm, _11xz.nvd, _11xz.nvm, _11xz.si, _11xz_FSTOrd50_0.doc, _11xz_FSTOrd50_0.tbk, _11xz_FSTOrd50_0.tix, _11y4.fdt, _11y4.fdx, _11y4.fnm, _11y4.nvd, _11y4.nvm, _11y4.si, _11y4_FSTOrd50_0.doc, _11y4_FSTOrd50_0.tbk, _11y4_FSTOrd50_0.tix, _11y6.fdt, _11y6.fdx, _11y6.fnm, _11y6.nvd, _11y6.nvm, _11y6.si, _11y6_FSTOrd50_0.doc, _11y6_FSTOrd50_0.tbk, _11y6_FSTOrd50_0.tix, _11y7.cfe, _11y7.cfs, _11y7.si, _11y8.fdt, _11y8.fdx, _11y8.fnm, _11y8.nvd, _11y8.nvm, _11y8.si, _11y8_FSTOrd50_0.doc, _11y8_FSTOrd50_0.tbk, _11y8_FSTOrd50_0.tix, _11ya.fdt, _11ya.fdx, _11ya.fnm, _11ya.nvd, _11ya.nvm, _11ya.si, _11ya_FSTOrd50_0.doc, _11ya_FSTOrd50_0.tbk, _11ya_FSTOrd50_0.tix, _11yc.fdt, _11yc.fdx, _11yc.fnm, _11yc.nvd, _11yc.nvm, _11yc.si, _11yc_FSTOrd50_0.doc, _11yc_FSTOrd50_0.tbk, _11yc_FSTOrd50_0.tix, _11yd.fdt, _11yd.fdx, _11yd.fnm, _11yd.nvd, _11yd.nvm, _11yd.si, _11yd_FSTOrd50_0.doc, _11yd_FSTOrd50_0.tbk, _11yd_FSTOrd50_0.tix, _11ye.fdt, _11ye.fdx, _11ye.fnm, _11ye.nvd, _11ye.nvm, _11ye.si, _11ye_FSTOrd50_0.doc, _11ye_FSTOrd50_0.tbk, _11ye_FSTOrd50_0.tix, _126b.fdt, _126b.fdx, _126b.fnm, _126b.nvd, _126b.nvm, _126b.si, _126b_FSTOrd50_0.doc, _126b_FSTOrd50_0.tbk, _126b_FSTOrd50_0.tix, _126g.fdt, _126g.fdx, _126g.fnm, _126g.nvd, _126g.nvm, _126g.si, _126g_FSTOrd50_0.doc, _126g_FSTOrd50_0.tbk, _126g_FSTOrd50_0.tix, _126h.cfe, _126h.cfs, _126h.si, _126p.fdt, _126p.fdx, _126p.fnm, _126p.nvd, _126p.nvm, _126p.si, _126p_FSTOrd50_0.doc, _126p_FSTOrd50_0.tbk, _126p_FSTOrd50_0.tix, _126v.fdt, _126v.fdx, _126v.fnm, _126v.nvd, _126v.nvm, _126v.si, _126v_FSTOrd50_0.doc, _126v_FSTOrd50_0.tbk, _126v_FSTOrd50_0.tix, _1274.fdt, _1274.fdx, _1274.fnm, _1274.nvd, _1274.nvm, _1274.si, _1274_FSTOrd50_0.doc, _1274_FSTOrd50_0.tbk, _1274_FSTOrd50_0.tix, _1277.fdt, _1277.fdx, _1277.fnm, _1277.nvd, _1277.nvm, _1277.si, _1277_FSTOrd50_0.doc, _1277_FSTOrd50_0.tbk, _1277_FSTOrd50_0.tix, _1278.fdt, _1278.fdx, _1278.fnm, _1278.nvd, _1278.nvm, _1278.si, _1278_FSTOrd50_0.doc, _1278_FSTOrd50_0.tbk, _1278_FSTOrd50_0.tix, _1279.cfe, _1279.cfs, _1279.si, _127a.fdt, _127a.fdx, _127a.fnm, _127a.nvd, _127a.nvm, _127a.si, _127a_FSTOrd50_0.doc, _127a_FSTOrd50_0.tbk, _127a_FSTOrd50_0.tix, _127b.fdt, _127b.fdx, _127b.fnm, _127b.nvd, _127b.nvm, _127b.si, _127b_FSTOrd50_0.doc, _127b_FSTOrd50_0.tbk, _127b_FSTOrd50_0.tix, _127c.fdt, _127c.fdx, _127c.fnm, _127c.nvd, _127c.nvm, _127c.si, _127c_FSTOrd50_0.doc, _127c_FSTOrd50_0.tbk, _127c_FSTOrd50_0.tix, _127d.fdt, _127d.fdx, _127d.fnm, _127d.nvd, _127d.nvm, _127d.si, _127d_FSTOrd50_0.doc, _127d_FSTOrd50_0.tbk, _127d_FSTOrd50_0.tix, _127e.fdt, _127e.fdx, _127e.fnm, _127e.nvd, _127e.nvm, _127e.si, _127e_FSTOrd50_0.doc, _127e_FSTOrd50_0.tbk, _127e_FSTOrd50_0.tix, _127f.fdt, _127f.fdx, _127f.fnm, _127f.nvd, _127f.nvm, _127f.si, _127f_FSTOrd50_0.doc, _127f_FSTOrd50_0.tbk, _127f_FSTOrd50_0.tix, _127h.fdt, _127h.fdx, _127h.fnm, _127h.nvd, _127h.nvm, _127h.si, _127h_FSTOrd50_0.doc, _127h_FSTOrd50_0.tbk, _127h_FSTOrd50_0.tix, segments_2]}, {indexVersion=1447979710547,generation=3,filelist=[_11yg.cfe, _11yg.cfs, _11yg.si, _126b.fdt, _126b.fdx, _126b.fnm, _126b.nvd, _126b.nvm, _126b.si, _126b_FSTOrd50_0.doc, _126b_FSTOrd50_0.tbk, _126b_FSTOrd50_0.tix, _126g.fdt, _126g.fdx, _126g.fnm, _126g.nvd, _126g.nvm, _126g.si, _126g_FSTOrd50_0.doc, _126g_FSTOrd50_0.tbk, _126g_FSTOrd50_0.tix, _126p.fdt, _126p.fdx, _126p.fnm, _126p.nvd, _126p.nvm, _126p.si, _126p_FSTOrd50_0.doc, _126p_FSTOrd50_0.tbk, _126p_FSTOrd50_0.tix, _126v.fdt, _126v.fdx, _126v.fnm, _126v.nvd, _126v.nvm, _126v.si, _126v_FSTOrd50_0.doc, _126v_FSTOrd50_0.tbk, _126v_FSTOrd50_0.tix, _127f.fdt, _127f.fdx, _127f.fnm, _127f.nvd, _127f.nvm, _127f.si, _127f_FSTOrd50_0.doc, _127f_FSTOrd50_0.tbk, _127f_FSTOrd50_0.tix, _127g.cfe, _127g.cfs, _127g.si, _127h.fdt, _127h.fdx, _127h.fnm, _127h.nvd, _127h.nvm, _127h.si, _127h_FSTOrd50_0.doc, _127h_FSTOrd50_0.tbk, _127h_FSTOrd50_0.tix, segments_3]}]>
	at __randomizedtesting.SeedInfo.seed([A3790642C165612C:86AE1D72B12D6F2F]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.failNotEquals(Assert.java:647)
	at org.junit.Assert.assertEquals(Assert.java:128)
	at org.junit.Assert.assertEquals(Assert.java:147)
	at org.apache.solr.handler.TestReplicationHandler.doTestReplicateAfterCoreReload(TestReplicationHandler.java:1136)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 10653 lines...]
   [junit4] Suite: org.apache.solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_A3790642C165612C-001/init-core-data-001
   [junit4]   2> 431197 INFO  (SUITE-LeaderInitiatedRecoveryOnShardRestartTest-seed#[A3790642C165612C]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /_j/q
   [junit4]   2> 431201 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 431202 INFO  (Thread-4884) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 431202 INFO  (Thread-4884) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 431302 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.ZkTestServer start zk server on port:60407
   [junit4]   2> 431302 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 431303 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 431306 INFO  (zkCallback-283-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6f2b0e86 name:ZooKeeperConnection Watcher:127.0.0.1:60407 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 431306 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 431306 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 431306 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 431309 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 431310 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 431317 INFO  (zkCallback-284-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@34095bc name:ZooKeeperConnection Watcher:127.0.0.1:60407/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 431317 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 431318 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 431318 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 431320 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 431322 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 431324 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 431326 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 431327 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 431335 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 431335 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 431337 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 431338 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 431340 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 431340 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 431342 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 431342 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 431344 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 431344 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 431346 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 431346 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 431348 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 431349 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 431356 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 431356 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 431360 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 431361 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 431365 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 431365 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 431679 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_A3790642C165612C-001/control-001/cores/collection1
   [junit4]   2> 431680 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 431682 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@3be6eb6f{/_j/q,null,AVAILABLE}
   [junit4]   2> 431683 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.e.j.s.ServerConnector Started ServerConnector@7c65e893{HTTP/1.1}{127.0.0.1:59007}
   [junit4]   2> 431683 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.e.j.s.Server Started @435173ms
   [junit4]   2> 431683 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostPort=59007, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_A3790642C165612C-001/tempDir-001/control/data, hostContext=/_j/q, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_A3790642C165612C-001/control-001/cores}
   [junit4]   2> 431683 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@4f6de641
   [junit4]   2> 431683 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_A3790642C165612C-001/control-001/'
   [junit4]   2> 431684 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 431684 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 431706 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 431707 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 431710 INFO  (zkCallback-285-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@611d491d name:ZooKeeperConnection Watcher:127.0.0.1:60407/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 431710 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 431711 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 431713 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 431713 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_A3790642C165612C-001/control-001/solr.xml
   [junit4]   2> 431736 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_A3790642C165612C-001/control-001/cores
   [junit4]   2> 431736 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.CoreContainer New CoreContainer 844721785
   [junit4]   2> 431736 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_A3790642C165612C-001/control-001/]
   [junit4]   2> 431736 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_A3790642C165612C-001/control-001/lib
   [junit4]   2> 431736 WARN  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_A3790642C165612C-001/control-001/lib).
   [junit4]   2> 431754 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 431756 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 431757 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 431758 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 431758 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:60407/solr
   [junit4]   2> 431758 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 431758 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 431759 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 431774 INFO  (zkCallback-287-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@4670da75 name:ZooKeeperConnection Watcher:127.0.0.1:60407 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 431775 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 431776 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 431778 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [n:127.0.0.1:59007__j%2Fq    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 431779 INFO  (zkCallback-288-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@77166a8b name:ZooKeeperConnection Watcher:127.0.0.1:60407/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 431779 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [n:127.0.0.1:59007__j%2Fq    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 431786 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [n:127.0.0.1:59007__j%2Fq    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 431788 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [n:127.0.0.1:59007__j%2Fq    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 431795 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [n:127.0.0.1:59007__j%2Fq    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 431797 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [n:127.0.0.1:59007__j%2Fq    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 431800 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [n:127.0.0.1:59007__j%2Fq    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 431804 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [n:127.0.0.1:59007__j%2Fq    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 431806 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [n:127.0.0.1:59007__j%2Fq    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 431808 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [n:127.0.0.1:59007__j%2Fq    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 431810 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [n:127.0.0.1:59007__j%2Fq    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 431811 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [n:127.0.0.1:59007__j%2Fq    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:59007__j%2Fq
   [junit4]   2> 431811 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [n:127.0.0.1:59007__j%2Fq    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:59007__j%2Fq
   [junit4]   2> 431814 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [n:127.0.0.1:59007__j%2Fq    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 431816 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [n:127.0.0.1:59007__j%2Fq    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 431818 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [n:127.0.0.1:59007__j%2Fq    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 431819 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [n:127.0.0.1:59007__j%2Fq    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94894735722938372-127.0.0.1:59007__j%2Fq-n_0000000000
   [junit4]   2> 431819 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [n:127.0.0.1:59007__j%2Fq    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:59007__j%2Fq
   [junit4]   2> 431819 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [n:127.0.0.1:59007__j%2Fq    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 431821 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [n:127.0.0.1:59007__j%2Fq    ] o.a.s.c.Overseer Overseer (id=94894735722938372-127.0.0.1:59007__j%2Fq-n_0000000000) starting
   [junit4]   2> 431824 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [n:127.0.0.1:59007__j%2Fq    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 431835 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [n:127.0.0.1:59007__j%2Fq    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 431835 INFO  (OverseerCollectionConfigSetProcessor-94894735722938372-127.0.0.1:59007__j%2Fq-n_0000000000) [n:127.0.0.1:59007__j%2Fq    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 431836 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [n:127.0.0.1:59007__j%2Fq    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 431838 INFO  (OverseerStateUpdate-94894735722938372-127.0.0.1:59007__j%2Fq-n_0000000000) [n:127.0.0.1:59007__j%2Fq    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 431862 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [n:127.0.0.1:59007__j%2Fq    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 431862 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [n:127.0.0.1:59007__j%2Fq    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 431863 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [n:127.0.0.1:59007__j%2Fq    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_A3790642C165612C-001/control-001/cores
   [junit4]   2> 431864 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [n:127.0.0.1:59007__j%2Fq    ] o.a.s.c.SolrCore Created CoreDescriptor: {dataDir=data/, config=solrconfig.xml, transient=false, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_A3790642C165612C-001/control-001/cores/collection1/, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_A3790642C165612C-001/control-001/cores/collection1, name=collection1, schema=schema.xml, configSetProperties=configsetprops.json, coreNodeName=, loadOnStartup=true, shard=, collection=control_collection}
   [junit4]   2> 431864 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [n:127.0.0.1:59007__j%2Fq    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_A3790642C165612C-001/control-001/cores/collection1/
   [junit4]   2> 431865 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [n:127.0.0.1:59007__j%2Fq    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 431867 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 431867 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 431875 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 431878 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [n:127.0.0.1:59007__j%2Fq    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 431878 INFO  (OverseerStateUpdate-94894735722938372-127.0.0.1:59007__j%2Fq-n_0000000000) [n:127.0.0.1:59007__j%2Fq    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "roles":null,
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:59007/_j/q",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "node_name":"127.0.0.1:59007__j%2Fq",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 431878 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [n:127.0.0.1:59007__j%2Fq    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 431879 INFO  (OverseerStateUpdate-94894735722938372-127.0.0.1:59007__j%2Fq-n_0000000000) [n:127.0.0.1:59007__j%2Fq    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "roles":null,
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:59007/_j/q",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "node_name":"127.0.0.1:59007__j%2Fq",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 431879 INFO  (OverseerStateUpdate-94894735722938372-127.0.0.1:59007__j%2Fq-n_0000000000) [n:127.0.0.1:59007__j%2Fq    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 431880 INFO  (OverseerStateUpdate-94894735722938372-127.0.0.1:59007__j%2Fq-n_0000000000) [n:127.0.0.1:59007__j%2Fq    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 431883 INFO  (zkCallback-288-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 432876 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 432876 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 432877 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 432877 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 432877 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 432878 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_A3790642C165612C-001/control-001/cores/collection1/'
   [junit4]   2> 432878 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 432878 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 432895 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 432902 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 432915 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 432944 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 432945 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 432952 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 433201 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 433209 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 433210 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 433222 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 433226 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 433229 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 433230 WARN  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 433230 WARN  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 433230 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 433231 WARN  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 433231 WARN  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 433232 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 433232 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 433232 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 433232 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_A3790642C165612C-001/control-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 433232 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@9eb620c
   [junit4]   2> 433233 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_A3790642C165612C-001/control-001/cores/collection1/data
   [junit4]   2> 433233 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_A3790642C165612C-001/control-001/cores/collection1/data/index/
   [junit4]   2> 433233 WARN  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_A3790642C165612C-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 433233 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_A3790642C165612C-001/control-001/cores/collection1/data/index
   [junit4]   2> 433233 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=4, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
   [junit4]   2> 433234 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@6e1d1ca4 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@39438b76),segFN=segments_1,generation=1}
   [junit4]   2> 433234 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 433235 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_A3790642C165612C-001/control-001/cores/collection1/data/
   [junit4]   2> 433235 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_A3790642C165612C-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 433239 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 433239 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 433240 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 433240 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 433240 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 433240 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 433240 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 433240 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 433240 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 433240 WARN  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
   [junit4]   2> 433243 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 433244 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 433246 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 433247 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 433251 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /replication,/admin/segments,/admin/mbeans,/admin/luke,/admin/system,/schema,/update,/admin/ping,/update/json,/config,/update/json/docs,standard,/get,/admin/properties,/admin/threads,/admin/plugins,/admin/file,/update/csv,/dump,/admin/logging
   [junit4]   2> 433252 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 433253 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 433253 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 433253 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 433253 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 433254 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=12, maxMergeAtOnceExplicit=32, maxMergedSegmentMB=0.271484375, floorSegmentMB=0.232421875, forceMergeDeletesPctAllowed=12.024019232950184, segmentsPerTier=24.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
   [junit4]   2> 433254 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@6e1d1ca4 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@39438b76),segFN=segments_1,generation=1}
   [junit4]   2> 433254 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 433255 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@46f2c8e2[collection1] main
   [junit4]   2> 433255 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 433256 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 433256 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 433256 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 433256 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 433256 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 433257 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 433257 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 433257 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 433257 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 433257 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 433257 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 433258 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 433258 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1518315773719740416
   [junit4]   2> 433258 INFO  (searcherExecutor-1131-thread-1-processing-x:collection1 s:shard1 r:core_node1 c:control_collection n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@46f2c8e2[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 433260 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 2.0ms to seed version buckets with highest version 1518315773719740416
   [junit4]   2> 433260 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 433261 INFO  (coreLoadExecutor-1130-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 433261 INFO  (coreZkRegister-1124-thread-1-processing-x:collection1 s:shard1 r:core_node1 c:control_collection n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:59007/_j/q collection:control_collection shard:shard1
   [junit4]   2> 433262 INFO  (coreZkRegister-1124-thread-1-processing-x:collection1 s:shard1 r:core_node1 c:control_collection n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 433266 INFO  (coreZkRegister-1124-thread-1-processing-x:collection1 s:shard1 r:core_node1 c:control_collection n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94894735722938372-core_node1-n_0000000000
   [junit4]   2> 433267 INFO  (coreZkRegister-1124-thread-1-processing-x:collection1 s:shard1 r:core_node1 c:control_collection n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 433269 INFO  (coreZkRegister-1124-thread-1-processing-x:collection1 s:shard1 r:core_node1 c:control_collection n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 433269 INFO  (OverseerStateUpdate-94894735722938372-127.0.0.1:59007__j%2Fq-n_0000000000) [n:127.0.0.1:59007__j%2Fq    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 433270 INFO  (coreZkRegister-1124-thread-1-processing-x:collection1 s:shard1 r:core_node1 c:control_collection n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 433270 INFO  (coreZkRegister-1124-thread-1-processing-x:collection1 s:shard1 r:core_node1 c:control_collection n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:59007/_j/q/collection1/
   [junit4]   2> 433270 INFO  (coreZkRegister-1124-thread-1-processing-x:collection1 s:shard1 r:core_node1 c:control_collection n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 433270 INFO  (coreZkRegister-1124-thread-1-processing-x:collection1 s:shard1 r:core_node1 c:control_collection n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:59007/_j/q/collection1/ has no replicas
   [junit4]   2> 433271 INFO  (coreZkRegister-1124-thread-1-processing-x:collection1 s:shard1 r:core_node1 c:control_collection n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 433274 INFO  (coreZkRegister-1124-thread-1-processing-x:collection1 s:shard1 r:core_node1 c:control_collection n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 433276 INFO  (coreZkRegister-1124-thread-1-processing-x:collection1 s:shard1 r:core_node1 c:control_collection n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:59007/_j/q/collection1/ shard1
   [junit4]   2> 433277 INFO  (OverseerStateUpdate-94894735722938372-127.0.0.1:59007__j%2Fq-n_0000000000) [n:127.0.0.1:59007__j%2Fq    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:59007/_j/q",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 1
   [junit4]   2> 433380 INFO  (zkCallback-288-thread-1-processing-n:127.0.0.1:59007__j%2Fq) [n:127.0.0.1:59007__j%2Fq    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 433380 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 433381 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 433382 INFO  (zkCallback-290-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6077f0ee name:ZooKeeperConnection Watcher:127.0.0.1:60407/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 433382 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 433382 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 433382 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 433385 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 433385 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with stateFormat=2
   [junit4]   2> 433385 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A3790642C165612C]) [    ] o.a.s.c.c.SolrZkClient Using defa

[...truncated too long message...]

/temp/solr.cloud.CollectionsAPIDistributedZkTest_A3790642C165612C-001/control-001/cores/awholynewstresscollection_collection5_0_shard2_replica1/data
   [junit4]   2> 17503051 INFO  (qtp62381361-15321) [n:127.0.0.1:40675_f_b%2Fx    ] o.a.s.c.CachingDirectoryFactory Removing directory after core close: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_A3790642C165612C-001/control-001/cores/awholynewstresscollection_collection5_0_shard2_replica1/data
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene54): {range_facet_l=PostingsFormat(name=LuceneFixedGap), id=Lucene50(blocksize=128), _version_=PostingsFormat(name=LuceneFixedGap), multiDefault=PostingsFormat(name=LuceneVarGapDocFreqInterval), a_i1=PostingsFormat(name=LuceneFixedGap), a_t=PostingsFormat(name=LuceneFixedGap), text=FSTOrd50, other_tl1=PostingsFormat(name=LuceneFixedGap), intDefault=PostingsFormat(name=LuceneFixedGap), timestamp=PostingsFormat(name=LuceneFixedGap), range_facet_i_dv=PostingsFormat(name=LuceneFixedGap), range_facet_l_dv=Lucene50(blocksize=128)}, docValues:{timestamp=DocValuesFormat(name=Memory), range_facet_i_dv=DocValuesFormat(name=Memory), range_facet_l_dv=DocValuesFormat(name=Lucene54)}, sim=RandomSimilarityProvider(queryNorm=false,coord=no): {}, locale=ar_QA, timezone=AGT
   [junit4]   2> 17479997 INFO  (qtp62381361-15372) [n:127.0.0.1:40675_f_b%2Fx    ] o.a.s.c.CachingDirectoryFactory looking to close /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_A3790642C165612C-001/control-001/cores/awholynewstresscollection_collection5_0_shard3_replica2/data [CachedDir<<refCount=0;path=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_A3790642C165612C-001/control-001/cores/awholynewstresscollection_collection5_0_shard3_replica2/data;done=false>>]
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_80 (64-bit)/cpus=4,threads=284,free=40079432,total=495976448
   [junit4]   2> NOTE: All tests run in this JVM: [DistanceUnitsTest, HdfsSyncSliceTest, TestZkChroot, TestMissingGroups, LeaderElectionTest, TestSearchPerf, IgnoreCommitOptimizeUpdateProcessorFactoryTest, SaslZkACLProviderTest, ClusterStateTest, TestCloudManagedSchema, TestCopyFieldCollectionResource, TestManagedSchemaDynamicFieldResource, AsyncCallRequestStatusResponseTest, TestSolrQueryParserDefaultOperatorResource, DateMathParserTest, BadComponentTest, SampleTest, TestPostingsSolrHighlighter, TestDownShardTolerantSearch, TestCollationField, TestReversedWildcardFilterFactory, RecoveryAfterSoftCommitTest, QueryResultKeyTest, DateRangeFieldTest, TestStressRecovery, TestReload, TestCoreDiscovery, DocExpirationUpdateProcessorFactoryTest, DocValuesMissingTest, BJQParserTest, LukeRequestHandlerTest, OverseerTaskQueueTest, DistributedFacetPivotSmallTest, TestConfigSetsAPIZkFailure, PreAnalyzedUpdateProcessorTest, DirectSolrConnectionTest, TestJettySolrRunner, SolrCoreCheckLockOnStartupTest, ShowFileRequestHandlerTest, LeaderInitiatedRecoveryOnShardRestartTest, TestFuzzyAnalyzedSuggestions, TestGroupingSearch, ScriptEngineTest, TestRuleBasedAuthorizationPlugin, TestSolrIndexConfig, TestSolrDeletionPolicy1, OverseerTest, TestSolrDynamicMBean, DirectUpdateHandlerTest, TestCSVResponseWriter, TestBlendedInfixSuggestions, SuggestComponentContextFilterQueryTest, CoreAdminHandlerTest, TestHashQParserPlugin, TestSolrCloudWithKerberosAlt, TestConfigSetImmutable, AutoCommitTest, BadIndexSchemaTest, CursorMarkTest, VMParamsZkACLAndCredentialsProvidersTest, TestSha256AuthenticationProvider, CircularListTest, TestSortingResponseWriter, TestIndexSearcher, CurrencyFieldXmlFileTest, TestCharFilters, TestBlobHandler, HdfsThreadLeakTest, TestElisionMultitermQuery, SuggesterTest, TestAddFieldRealTimeGet, TestAuthenticationFramework, TestOverriddenPrefixQueryForCustomFieldType, FacetPivotSmallTest, TestSweetSpotSimilarityFactory, StatsComponentTest, CollectionStateFormat2Test, PrimitiveFieldTypeTest, TestUtils, PluginInfoTest, TestReloadDeadlock, TestMiniSolrCloudCluster, UpdateParamsTest, TestConfigOverlay, DocValuesMultiTest, DeleteLastCustomShardedReplicaTest, BasicDistributedZkTest, BasicDistributedZk2Test, UnloadDistributedZkTest, CollectionsAPIDistributedZkTest]
   [junit4]   2> 17505815 INFO  (qtp62381361-15372) [n:127.0.0.1:40675_f_b%2Fx    ] o.a.s.c.CachingDirectoryFactory looking to close /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_A3790642C165612C-001/control-001/cores/awholynewstresscollection_collection5_0_shard3_replica2/data []
   [junit4]   2> 17479161 INFO  (qtp47016225-14852) [n:127.0.0.1:48316_f_b%2Fx c:awholynewstresscollection_collection2_0 s:shard5 r:core_node9 x:awholynewstresscollection_collection2_0_shard5_replica3] o.a.s.s.SolrDispatchFilter [admin] webapp=null path=/admin/cores params={qt=/admin/cores&action=CREATE&numShards=5&shard=shard5&version=2&collection=awholynewstresscollection_collection2_0&name=awholynewstresscollection_collection2_0_shard5_replica3&collection.configName=conf1&wt=javabin} status=0 QTime=15459256 
   [junit4]   2> 17479160 INFO  (qtp1482630108-14938) [n:127.0.0.1:52329_f_b%2Fx c:awholynewstresscollection_collection3_0 s:shard2 r:core_node18 x:awholynewstresscollection_collection3_0_shard2_replica1] o.a.s.s.SolrDispatchFilter [admin] webapp=null path=/admin/cores params={qt=/admin/cores&action=CREATE&numShards=5&shard=shard2&version=2&collection=awholynewstresscollection_collection3_0&name=awholynewstresscollection_collection3_0_shard2_replica1&collection.configName=conf1&wt=javabin} status=0 QTime=15458920 
   [junit4]   2> 17479160 INFO  (qtp1482630108-14879) [n:127.0.0.1:52329_f_b%2Fx c:awholynewstresscollection_collection2_0 s:shard1 r:core_node11 x:awholynewstresscollection_collection2_0_shard1_replica2] o.a.s.s.SolrDispatchFilter [admin] webapp=null path=/admin/cores params={qt=/admin/cores&action=CREATE&numShards=5&shard=shard1&version=2&collection=awholynewstresscollection_collection2_0&name=awholynewstresscollection_collection2_0_shard1_replica2&collection.configName=conf1&wt=javabin} status=0 QTime=15450581 
   [junit4]   2> 17479160 INFO  (qtp47016225-14927) [n:127.0.0.1:48316_f_b%2Fx    ] o.a.s.c.CachingDirectoryFactory Closing MockDirectoryFactory - 2 directories currently being tracked
   [junit4]   2> 17505816 INFO  (qtp47016225-14927) [n:127.0.0.1:48316_f_b%2Fx    ] o.a.s.c.CachingDirectoryFactory looking to close /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_A3790642C165612C-001/shard-1-001/cores/awholynewstresscollection_collection5_0_shard3_replica1/data [CachedDir<<refCount=0;path=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_A3790642C165612C-001/shard-1-001/cores/awholynewstresscollection_collection5_0_shard3_replica1/data;done=false>>]
   [junit4]   2> 17479160 INFO  (qtp1302440199-14671) [n:127.0.0.1:32829_f_b%2Fx    ] o.a.s.c.CachingDirectoryFactory Closing MockDirectoryFactory - 2 directories currently being tracked
   [junit4] ERROR   0.00s J2 | CollectionsAPIDistributedZkTest (suite) <<<
   [junit4]    > Throwable #1: java.lang.OutOfMemoryError: Java heap space
   [junit4] Completed [548/548] on J2 in 15569.37s, 1 test, 1 error <<< FAILURES!

[...truncated 1 lines...]
   [junit4] JVM J2: stdout was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J2-20151120_001200_739.sysout
   [junit4] >>> JVM J2: stdout (verbatim) ----
   [junit4] java.lang.OutOfMemoryError: Java heap space
   [junit4] Dumping heap to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/heapdumps/java_pid18208.hprof ...
   [junit4] Heap dump file created [676079728 bytes in 48.389 secs]
   [junit4] <<< JVM J2: EOF ----

   [junit4] JVM J2: stderr was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J2-20151120_001200_739.syserr
   [junit4] >>> JVM J2: stderr (verbatim) ----
   [junit4] 
   [junit4] Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "TEST-CollectionsAPIDistributedZkTest.test-seed#[A3790642C165612C]-SendThread(127.0.0.1:57818)"
   [junit4] <<< JVM J2: EOF ----

[...truncated 12 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 548 suites (7 ignored), 2165 tests, 1 suite-level error, 2 failures, 32 ignored (18 assumptions) [seed: A3790642C165612C]

Total time: 416 minutes 3 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1021 - Still Failing

[Apache Jenkins Server <je...@builds.apache.org>]

Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1021/

2 tests failed.
FAILED:  org.apache.solr.hadoop.MorphlineBasicMiniMRTest.mrRun

Error Message:
Failed on local exception: java.io.IOException: Connection reset by peer; Host Details : local host is: "lucene1-us-west/10.41.0.5"; destination host is: "lucene1-us-west.apache.org":37722; 

Stack Trace:
java.io.IOException: Failed on local exception: java.io.IOException: Connection reset by peer; Host Details : local host is: "lucene1-us-west/10.41.0.5"; destination host is: "lucene1-us-west.apache.org":37722; 
	at __randomizedtesting.SeedInfo.seed([769F8DA6084321B3:78CD39A809D513BC]:0)
	at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:772)
	at org.apache.hadoop.ipc.Client.call(Client.java:1472)
	at org.apache.hadoop.ipc.Client.call(Client.java:1399)
	at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:232)
	at com.sun.proxy.$Proxy108.getClusterMetrics(Unknown Source)
	at org.apache.hadoop.yarn.api.impl.pb.client.ApplicationClientProtocolPBClientImpl.getClusterMetrics(ApplicationClientProtocolPBClientImpl.java:202)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:187)
	at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)
	at com.sun.proxy.$Proxy109.getClusterMetrics(Unknown Source)
	at org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.getYarnClusterMetrics(YarnClientImpl.java:461)
	at org.apache.hadoop.mapred.ResourceMgrDelegate.getClusterMetrics(ResourceMgrDelegate.java:151)
	at org.apache.hadoop.mapred.YARNRunner.getClusterMetrics(YARNRunner.java:179)
	at org.apache.hadoop.mapreduce.Cluster.getClusterStatus(Cluster.java:246)
	at org.apache.hadoop.mapred.JobClient$3.run(JobClient.java:719)
	at org.apache.hadoop.mapred.JobClient$3.run(JobClient.java:717)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
	at org.apache.hadoop.mapred.JobClient.getClusterStatus(JobClient.java:717)
	at org.apache.solr.hadoop.MapReduceIndexerTool.run(MapReduceIndexerTool.java:644)
	at org.apache.solr.hadoop.MapReduceIndexerTool.run(MapReduceIndexerTool.java:607)
	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
	at org.apache.solr.hadoop.MorphlineBasicMiniMRTest.mrRun(MorphlineBasicMiniMRTest.java:363)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.IOException: Connection reset by peer
	at sun.nio.ch.FileDispatcherImpl.read0(Native Method)
	at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
	at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223)
	at sun.nio.ch.IOUtil.read(IOUtil.java:197)
	at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:384)
	at org.apache.hadoop.net.SocketInputStream$Reader.performIO(SocketInputStream.java:57)
	at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:142)
	at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161)
	at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131)
	at java.io.FilterInputStream.read(FilterInputStream.java:133)
	at java.io.FilterInputStream.read(FilterInputStream.java:133)
	at org.apache.hadoop.ipc.Client$Connection$PingInputStream.read(Client.java:513)
	at java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
	at java.io.BufferedInputStream.read(BufferedInputStream.java:254)
	at java.io.DataInputStream.readInt(DataInputStream.java:387)
	at org.apache.hadoop.ipc.Client$Connection.receiveRpcResponse(Client.java:1071)
	at org.apache.hadoop.ipc.Client$Connection.run(Client.java:966)


FAILED:  org.apache.solr.hadoop.MorphlineGoLiveMiniMRTest.test

Error Message:
Failed on local exception: java.io.IOException: Connection reset by peer; Host Details : local host is: "lucene1-us-west/10.41.0.5"; destination host is: "lucene1-us-west.apache.org":44672; 

Stack Trace:
java.io.IOException: Failed on local exception: java.io.IOException: Connection reset by peer; Host Details : local host is: "lucene1-us-west/10.41.0.5"; destination host is: "lucene1-us-west.apache.org":44672; 
	at __randomizedtesting.SeedInfo.seed([769F8DA6084321B3:FECBB27CA6BF4C4B]:0)
	at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:772)
	at org.apache.hadoop.ipc.Client.call(Client.java:1472)
	at org.apache.hadoop.ipc.Client.call(Client.java:1399)
	at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:232)
	at com.sun.proxy.$Proxy110.getClusterMetrics(Unknown Source)
	at org.apache.hadoop.yarn.api.impl.pb.client.ApplicationClientProtocolPBClientImpl.getClusterMetrics(ApplicationClientProtocolPBClientImpl.java:202)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:187)
	at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)
	at com.sun.proxy.$Proxy111.getClusterMetrics(Unknown Source)
	at org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.getYarnClusterMetrics(YarnClientImpl.java:461)
	at org.apache.hadoop.mapred.ResourceMgrDelegate.getClusterMetrics(ResourceMgrDelegate.java:151)
	at org.apache.hadoop.mapred.YARNRunner.getClusterMetrics(YARNRunner.java:179)
	at org.apache.hadoop.mapreduce.Cluster.getClusterStatus(Cluster.java:246)
	at org.apache.hadoop.mapred.JobClient$3.run(JobClient.java:719)
	at org.apache.hadoop.mapred.JobClient$3.run(JobClient.java:717)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
	at org.apache.hadoop.mapred.JobClient.getClusterStatus(JobClient.java:717)
	at org.apache.solr.hadoop.MapReduceIndexerTool.run(MapReduceIndexerTool.java:644)
	at org.apache.solr.hadoop.MapReduceIndexerTool.run(MapReduceIndexerTool.java:607)
	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
	at org.apache.solr.hadoop.MorphlineGoLiveMiniMRTest.test(MorphlineGoLiveMiniMRTest.java:402)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:963)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:938)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.IOException: Connection reset by peer
	at sun.nio.ch.FileDispatcherImpl.read0(Native Method)
	at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
	at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223)
	at sun.nio.ch.IOUtil.read(IOUtil.java:197)
	at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:384)
	at org.apache.hadoop.net.SocketInputStream$Reader.performIO(SocketInputStream.java:57)
	at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:142)
	at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161)
	at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131)
	at java.io.FilterInputStream.read(FilterInputStream.java:133)
	at java.io.FilterInputStream.read(FilterInputStream.java:133)
	at org.apache.hadoop.ipc.Client$Connection$PingInputStream.read(Client.java:513)
	at java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
	at java.io.BufferedInputStream.read(BufferedInputStream.java:254)
	at java.io.DataInputStream.readInt(DataInputStream.java:387)
	at org.apache.hadoop.ipc.Client$Connection.receiveRpcResponse(Client.java:1071)
	at org.apache.hadoop.ipc.Client$Connection.run(Client.java:966)




Build Log:
[...truncated 18076 lines...]
   [junit4] Suite: org.apache.solr.hadoop.MorphlineBasicMiniMRTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/init-core-data-001
   [junit4]   2> 0    INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true)
   [junit4]   2> 1153 WARN  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.u.NativeCodeLoader Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
   [junit4]   1> Formatting using clusterid: testClusterID
   [junit4]   2> 1324 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.n.FSNamesystem No KeyProvider found.
   [junit4]   2> 1373 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.n.FSNamesystem fsLock is fair:true
   [junit4]   2> 1460 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.c.C.deprecation hadoop.configured.node.mapping is deprecated. Instead, use net.topology.configured.node.mapping
   [junit4]   2> 1460 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.b.DatanodeManager dfs.block.invalidate.limit=1000
   [junit4]   2> 1461 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.b.DatanodeManager dfs.namenode.datanode.registration.ip-hostname-check=true
   [junit4]   2> 1465 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.b.BlockManager dfs.namenode.startup.delay.block.deletion.sec is set to 000:00:00:00.000
   [junit4]   2> 1468 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.b.BlockManager The block deletion will start around 2015 nov 19 02:00:15
   [junit4]   2> 1473 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.u.GSet Computing capacity for map BlocksMap
   [junit4]   2> 1473 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.u.GSet VM type       = 64-bit
   [junit4]   2> 1476 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.u.GSet 2.0% max memory 455 MB = 9.1 MB
   [junit4]   2> 1476 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.u.GSet capacity      = 2^20 = 1048576 entries
   [junit4]   2> 1511 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.b.BlockManager dfs.block.access.token.enable=false
   [junit4]   2> 1512 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.b.BlockManager defaultReplication         = 2
   [junit4]   2> 1512 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.b.BlockManager maxReplication             = 512
   [junit4]   2> 1512 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.b.BlockManager minReplication             = 1
   [junit4]   2> 1512 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.b.BlockManager maxReplicationStreams      = 2
   [junit4]   2> 1512 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.b.BlockManager shouldCheckForEnoughRacks  = false
   [junit4]   2> 1513 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.b.BlockManager replicationRecheckInterval = 3000
   [junit4]   2> 1513 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.b.BlockManager encryptDataTransfer        = false
   [junit4]   2> 1513 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.b.BlockManager maxNumBlocksToLog          = 1000
   [junit4]   2> 1522 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.n.FSNamesystem fsOwner             = jenkins (auth:SIMPLE)
   [junit4]   2> 1523 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.n.FSNamesystem supergroup          = supergroup
   [junit4]   2> 1523 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.n.FSNamesystem isPermissionEnabled = true
   [junit4]   2> 1523 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.n.FSNamesystem HA Enabled: false
   [junit4]   2> 1529 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.n.FSNamesystem Append Enabled: true
   [junit4]   2> 1818 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.u.GSet Computing capacity for map INodeMap
   [junit4]   2> 1818 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.u.GSet VM type       = 64-bit
   [junit4]   2> 1818 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.u.GSet 1.0% max memory 455 MB = 4.5 MB
   [junit4]   2> 1819 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.u.GSet capacity      = 2^19 = 524288 entries
   [junit4]   2> 1819 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.n.NameNode Caching file names occuring more than 10 times
   [junit4]   2> 1835 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.u.GSet Computing capacity for map cachedBlocks
   [junit4]   2> 1835 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.u.GSet VM type       = 64-bit
   [junit4]   2> 1835 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.u.GSet 0.25% max memory 455 MB = 1.1 MB
   [junit4]   2> 1836 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.u.GSet capacity      = 2^17 = 131072 entries
   [junit4]   2> 1838 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.n.FSNamesystem dfs.namenode.safemode.threshold-pct = 0.9990000128746033
   [junit4]   2> 1838 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.n.FSNamesystem dfs.namenode.safemode.min.datanodes = 0
   [junit4]   2> 1838 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.n.FSNamesystem dfs.namenode.safemode.extension     = 0
   [junit4]   2> 1841 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.n.FSNamesystem Retry cache on namenode is enabled
   [junit4]   2> 1842 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.n.FSNamesystem Retry cache will use 0.03 of total heap and retry cache entry expiry time is 600000 millis
   [junit4]   2> 1847 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.u.GSet Computing capacity for map NameNodeRetryCache
   [junit4]   2> 1848 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.u.GSet VM type       = 64-bit
   [junit4]   2> 1848 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.u.GSet 0.029999999329447746% max memory 455 MB = 139.8 KB
   [junit4]   2> 1848 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.u.GSet capacity      = 2^14 = 16384 entries
   [junit4]   2> 1858 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.n.NNConf ACLs enabled? false
   [junit4]   2> 1859 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.n.NNConf XAttrs enabled? true
   [junit4]   2> 1859 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.n.NNConf Maximum size of an xattr: 16384
   [junit4]   2> 1963 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.n.FSImage Allocated new BlockPoolId: BP-1156588151-10.41.0.5-1447923616226
   [junit4]   2> 1988 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.c.Storage Storage directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/name1 has been successfully formatted.
   [junit4]   2> 1993 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.c.Storage Storage directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/name2 has been successfully formatted.
   [junit4]   2> 2266 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.n.NNStorageRetentionManager Going to retain 1 images with txid >= 0
   [junit4]   2> 2270 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.n.NameNode createNameNode []
   [junit4]   2> 2342 WARN  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
   [junit4]   2> 2441 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.m.i.MetricsSystemImpl Scheduled snapshot period at 10 second(s).
   [junit4]   2> 2441 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.m.i.MetricsSystemImpl NameNode metrics system started
   [junit4]   2> 2445 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.n.NameNode fs.defaultFS is hdfs://127.0.0.1:0
   [junit4]   2> 2538 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.DFSUtil Starting Web-server for hdfs at: http://localhost:0
   [junit4]   2> 2713 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.m.log Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
   [junit4]   2> 2744 WARN  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 2778 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.HttpServer2 Added global filter 'safety' (class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter)
   [junit4]   2> 2783 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.HttpServer2 Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context hdfs
   [junit4]   2> 2787 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.HttpServer2 Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs
   [junit4]   2> 2787 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.HttpServer2 Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context static
   [junit4]   2> 2863 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.HttpServer2 Added filter 'org.apache.hadoop.hdfs.web.AuthFilter' (class=org.apache.hadoop.hdfs.web.AuthFilter)
   [junit4]   2> 2866 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.HttpServer2 addJerseyResourcePackage: packageName=org.apache.hadoop.hdfs.server.namenode.web.resources;org.apache.hadoop.hdfs.web.resources, pathSpec=/webhdfs/v1/*
   [junit4]   2> 2894 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.HttpServer2 Jetty bound to port 36968
   [junit4]   2> 2894 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 2977 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/hdfs to ./temp/Jetty_localhost_36968_hdfs____xbxi5e/webapp
   [junit4]   2> 3207 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 4270 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:36968
   [junit4]   2> 4282 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.n.FSNamesystem No KeyProvider found.
   [junit4]   2> 4283 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.n.FSNamesystem fsLock is fair:true
   [junit4]   2> 4284 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.b.DatanodeManager dfs.block.invalidate.limit=1000
   [junit4]   2> 4284 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.b.DatanodeManager dfs.namenode.datanode.registration.ip-hostname-check=true
   [junit4]   2> 4284 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.b.BlockManager dfs.namenode.startup.delay.block.deletion.sec is set to 000:00:00:00.000
   [junit4]   2> 4285 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.b.BlockManager The block deletion will start around 2015 nov 19 02:00:18
   [junit4]   2> 4285 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.u.GSet Computing capacity for map BlocksMap
   [junit4]   2> 4285 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.u.GSet VM type       = 64-bit
   [junit4]   2> 4285 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.u.GSet 2.0% max memory 455 MB = 9.1 MB
   [junit4]   2> 4285 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.u.GSet capacity      = 2^20 = 1048576 entries
   [junit4]   2> 4303 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.b.BlockManager dfs.block.access.token.enable=false
   [junit4]   2> 4303 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.b.BlockManager defaultReplication         = 2
   [junit4]   2> 4303 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.b.BlockManager maxReplication             = 512
   [junit4]   2> 4303 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.b.BlockManager minReplication             = 1
   [junit4]   2> 4303 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.b.BlockManager maxReplicationStreams      = 2
   [junit4]   2> 4304 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.b.BlockManager shouldCheckForEnoughRacks  = false
   [junit4]   2> 4304 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.b.BlockManager replicationRecheckInterval = 3000
   [junit4]   2> 4304 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.b.BlockManager encryptDataTransfer        = false
   [junit4]   2> 4304 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.b.BlockManager maxNumBlocksToLog          = 1000
   [junit4]   2> 4304 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.n.FSNamesystem fsOwner             = jenkins (auth:SIMPLE)
   [junit4]   2> 4304 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.n.FSNamesystem supergroup          = supergroup
   [junit4]   2> 4305 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.n.FSNamesystem isPermissionEnabled = true
   [junit4]   2> 4305 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.n.FSNamesystem HA Enabled: false
   [junit4]   2> 4305 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.n.FSNamesystem Append Enabled: true
   [junit4]   2> 4306 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.u.GSet Computing capacity for map INodeMap
   [junit4]   2> 4306 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.u.GSet VM type       = 64-bit
   [junit4]   2> 4306 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.u.GSet 1.0% max memory 455 MB = 4.5 MB
   [junit4]   2> 4307 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.u.GSet capacity      = 2^19 = 524288 entries
   [junit4]   2> 4307 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.n.NameNode Caching file names occuring more than 10 times
   [junit4]   2> 4307 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.u.GSet Computing capacity for map cachedBlocks
   [junit4]   2> 4307 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.u.GSet VM type       = 64-bit
   [junit4]   2> 4308 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.u.GSet 0.25% max memory 455 MB = 1.1 MB
   [junit4]   2> 4308 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.u.GSet capacity      = 2^17 = 131072 entries
   [junit4]   2> 4308 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.n.FSNamesystem dfs.namenode.safemode.threshold-pct = 0.9990000128746033
   [junit4]   2> 4308 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.n.FSNamesystem dfs.namenode.safemode.min.datanodes = 0
   [junit4]   2> 4308 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.n.FSNamesystem dfs.namenode.safemode.extension     = 0
   [junit4]   2> 4309 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.n.FSNamesystem Retry cache on namenode is enabled
   [junit4]   2> 4309 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.n.FSNamesystem Retry cache will use 0.03 of total heap and retry cache entry expiry time is 600000 millis
   [junit4]   2> 4309 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.u.GSet Computing capacity for map NameNodeRetryCache
   [junit4]   2> 4309 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.u.GSet VM type       = 64-bit
   [junit4]   2> 4309 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.u.GSet 0.029999999329447746% max memory 455 MB = 139.8 KB
   [junit4]   2> 4310 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.u.GSet capacity      = 2^14 = 16384 entries
   [junit4]   2> 4312 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.n.NNConf ACLs enabled? false
   [junit4]   2> 4313 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.n.NNConf XAttrs enabled? true
   [junit4]   2> 4313 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.n.NNConf Maximum size of an xattr: 16384
   [junit4]   2> 4321 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.c.Storage Lock on /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/name1/in_use.lock acquired by nodename 14467@lucene1-us-west
   [junit4]   2> 4327 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.c.Storage Lock on /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/name2/in_use.lock acquired by nodename 14467@lucene1-us-west
   [junit4]   2> 4332 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.n.FileJournalManager Recovering unfinalized segments in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/name1/current
   [junit4]   2> 4332 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.n.FileJournalManager Recovering unfinalized segments in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/name2/current
   [junit4]   2> 4332 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.n.FSImage No edit log streams selected.
   [junit4]   2> 4359 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.n.FSImageFormatPBINode Loading 1 INodes.
   [junit4]   2> 4374 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.n.FSImageFormatProtobuf Loaded FSImage in 0 seconds.
   [junit4]   2> 4374 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.n.FSImage Loaded image for txid 0 from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/name1/current/fsimage_0000000000000000000
   [junit4]   2> 4382 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.n.FSNamesystem Need to save fs image? false (staleImage=false, haEnabled=false, isRollingUpgrade=false)
   [junit4]   2> 4382 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.n.FSEditLog Starting log segment at 1
   [junit4]   2> 4435 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.n.NameCache initialized with 0 entries 0 lookups
   [junit4]   2> 4435 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.n.FSNamesystem Finished loading FSImage in 122 msecs
   [junit4]   2> 4850 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.n.NameNode RPC server is binding to localhost:0
   [junit4]   2> 4865 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.i.CallQueueManager Using callQueue class java.util.concurrent.LinkedBlockingQueue
   [junit4]   2> 4893 INFO  (Socket Reader #1 for port 33481) [    ] o.a.h.i.Server Starting Socket Reader #1 for port 33481
   [junit4]   2> 4945 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.n.NameNode Clients are to use localhost:33481 to access this namenode/service.
   [junit4]   2> 4952 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.n.FSNamesystem Registered FSNamesystemState MBean
   [junit4]   2> 4991 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.n.FSNamesystem Number of blocks under construction: 0
   [junit4]   2> 4991 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.n.FSNamesystem Number of blocks under construction: 0
   [junit4]   2> 4992 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.n.FSNamesystem initializing replication queues
   [junit4]   2> 4992 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.StateChange STATE* Leaving safe mode after 0 secs
   [junit4]   2> 4993 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.StateChange STATE* Network topology has 0 racks and 0 datanodes
   [junit4]   2> 4993 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.StateChange STATE* UnderReplicatedBlocks has 0 blocks
   [junit4]   2> 4998 INFO  (Replication Queue Initializer) [    ] o.a.h.h.s.b.BlockManager Total number of blocks            = 0
   [junit4]   2> 5006 INFO  (Replication Queue Initializer) [    ] o.a.h.h.s.b.BlockManager Number of invalid blocks          = 0
   [junit4]   2> 5006 INFO  (Replication Queue Initializer) [    ] o.a.h.h.s.b.BlockManager Number of under-replicated blocks = 0
   [junit4]   2> 5007 INFO  (Replication Queue Initializer) [    ] o.a.h.h.s.b.BlockManager Number of  over-replicated blocks = 0
   [junit4]   2> 5007 INFO  (Replication Queue Initializer) [    ] o.a.h.h.s.b.BlockManager Number of blocks being written    = 0
   [junit4]   2> 5007 INFO  (Replication Queue Initializer) [    ] o.a.h.h.StateChange STATE* Replication Queue initialization scan for invalid, over- and under-replicated blocks completed in 15 msec
   [junit4]   2> 5094 INFO  (IPC Server Responder) [    ] o.a.h.i.Server IPC Server Responder: starting
   [junit4]   2> 5104 INFO  (IPC Server listener on 33481) [    ] o.a.h.i.Server IPC Server listener on 33481: starting
   [junit4]   2> 5108 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.n.NameNode NameNode RPC up at: localhost/127.0.0.1:33481
   [junit4]   2> 5108 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.n.FSNamesystem Starting services required for active state
   [junit4]   2> 5121 INFO  (CacheReplicationMonitor(1187388745)) [    ] o.a.h.h.s.b.CacheReplicationMonitor Starting CacheReplicationMonitor with interval 30000 milliseconds
   [junit4]   2> 5121 INFO  (CacheReplicationMonitor(1187388745)) [    ] o.a.h.h.s.b.CacheReplicationMonitor Rescanning after 16383917487 milliseconds
   [junit4]   2> 5124 INFO  (CacheReplicationMonitor(1187388745)) [    ] o.a.h.h.s.b.CacheReplicationMonitor Scanned 0 directive(s) and 0 block(s) in 3 millisecond(s).
   [junit4]   2> 5132 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.MiniDFSCluster Starting DataNode 0 with dfs.datanode.data.dir: [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data1,[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data2
   [junit4]   2> 5225 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.m.i.MetricsSystemImpl DataNode metrics system started (again)
   [junit4]   2> 5227 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.d.DataNode Configured hostname is 127.0.0.1
   [junit4]   2> 5236 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.d.DataNode Starting DataNode with maxLockedMemory = 0
   [junit4]   2> 5245 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.d.DataNode Opened streaming server at /127.0.0.1:53833
   [junit4]   2> 5250 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.d.DataNode Balancing bandwith is 1048576 bytes/s
   [junit4]   2> 5250 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.d.DataNode Number threads for balancing is 5
   [junit4]   2> 5257 WARN  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 5258 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.HttpServer2 Added global filter 'safety' (class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter)
   [junit4]   2> 5259 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.HttpServer2 Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context datanode
   [junit4]   2> 5259 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.HttpServer2 Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context static
   [junit4]   2> 5259 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.HttpServer2 Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs
   [junit4]   2> 5266 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.HttpServer2 addJerseyResourcePackage: packageName=org.apache.hadoop.hdfs.server.datanode.web.resources;org.apache.hadoop.hdfs.web.resources, pathSpec=/webhdfs/v1/*
   [junit4]   2> 5267 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.HttpServer2 Jetty bound to port 37520
   [junit4]   2> 5267 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 5288 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_37520_datanode____.vfq9u2/webapp
   [junit4]   2> 5475 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 5953 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:37520
   [junit4]   2> 5956 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.d.DataNode dnUserName = jenkins
   [junit4]   2> 5956 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.d.DataNode supergroup = supergroup
   [junit4]   2> 5984 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.i.CallQueueManager Using callQueue class java.util.concurrent.LinkedBlockingQueue
   [junit4]   2> 5984 INFO  (Socket Reader #1 for port 38809) [    ] o.a.h.i.Server Starting Socket Reader #1 for port 38809
   [junit4]   2> 5996 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.d.DataNode Opened IPC server at /127.0.0.1:38809
   [junit4]   2> 6010 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.d.DataNode Refresh request received for nameservices: null
   [junit4]   2> 6014 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.d.DataNode Starting BPOfferServices for nameservices: <default>
   [junit4]   2> 6035 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:33481) [    ] o.a.h.h.s.d.DataNode Block pool <registering> (Datanode Uuid unassigned) service to localhost/127.0.0.1:33481 starting to offer service
   [junit4]   2> 6062 INFO  (IPC Server Responder) [    ] o.a.h.i.Server IPC Server Responder: starting
   [junit4]   2> 6067 INFO  (IPC Server listener on 38809) [    ] o.a.h.i.Server IPC Server listener on 38809: starting
   [junit4]   2> 6081 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.MiniDFSCluster Starting DataNode 1 with dfs.datanode.data.dir: [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data3,[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data4
   [junit4]   2> 6120 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.m.i.MetricsSystemImpl DataNode metrics system started (again)
   [junit4]   2> 6120 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.d.DataNode Configured hostname is 127.0.0.1
   [junit4]   2> 6121 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.d.DataNode Starting DataNode with maxLockedMemory = 0
   [junit4]   2> 6122 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.d.DataNode Opened streaming server at /127.0.0.1:35873
   [junit4]   2> 6122 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.d.DataNode Balancing bandwith is 1048576 bytes/s
   [junit4]   2> 6122 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.d.DataNode Number threads for balancing is 5
   [junit4]   2> 6125 WARN  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 6126 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.HttpServer2 Added global filter 'safety' (class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter)
   [junit4]   2> 6127 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.HttpServer2 Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context datanode
   [junit4]   2> 6127 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.HttpServer2 Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs
   [junit4]   2> 6127 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.HttpServer2 Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context static
   [junit4]   2> 6129 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.HttpServer2 addJerseyResourcePackage: packageName=org.apache.hadoop.hdfs.server.datanode.web.resources;org.apache.hadoop.hdfs.web.resources, pathSpec=/webhdfs/v1/*
   [junit4]   2> 6129 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.HttpServer2 Jetty bound to port 38511
   [junit4]   2> 6130 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 6166 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_38511_datanode____ynqhrr/webapp
   [junit4]   2> 6301 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 6733 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:38511
   [junit4]   2> 6734 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.d.DataNode dnUserName = jenkins
   [junit4]   2> 6734 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.d.DataNode supergroup = supergroup
   [junit4]   2> 6735 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.i.CallQueueManager Using callQueue class java.util.concurrent.LinkedBlockingQueue
   [junit4]   2> 6736 INFO  (Socket Reader #1 for port 41943) [    ] o.a.h.i.Server Starting Socket Reader #1 for port 41943
   [junit4]   2> 6740 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.d.DataNode Opened IPC server at /127.0.0.1:41943
   [junit4]   2> 6729 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:33481) [    ] o.a.h.h.s.c.Storage DataNode version: -56 and NameNode layout version: -60
   [junit4]   2> 6764 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.d.DataNode Refresh request received for nameservices: null
   [junit4]   2> 6764 INFO  (SUITE-MorphlineBasicMiniMRTest-seed#[769F8DA6084321B3]-worker) [    ] o.a.h.h.s.d.DataNode Starting BPOfferServices for nameservices: <default>
   [junit4]   2> 6766 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:33481) [    ] o.a.h.h.s.c.Storage Lock on /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data1/in_use.lock acquired by nodename 14467@lucene1-us-west
   [junit4]   2> 6769 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:33481) [    ] o.a.h.h.s.c.Storage Storage directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data1 is not formatted for BP-1156588151-10.41.0.5-1447923616226
   [junit4]   2> 6769 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:33481) [    ] o.a.h.h.s.c.Storage Formatting ...
   [junit4]   2> 6771 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:33481) [    ] o.a.h.h.s.d.DataNode Block pool <registering> (Datanode Uuid unassigned) service to localhost/127.0.0.1:33481 starting to offer service
   [junit4]   2> 6772 INFO  (IPC Server Responder) [    ] o.a.h.i.Server IPC Server Responder: starting
   [junit4]   2> 6772 INFO  (IPC Server listener on 41943) [    ] o.a.h.i.Server IPC Server listener on 41943: starting
   [junit4]   2> 6799 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:33481) [    ] o.a.h.h.s.c.Storage DataNode version: -56 and NameNode layout version: -60
   [junit4]   2> 6800 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:33481) [    ] o.a.h.h.s.c.Storage Lock on /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data2/in_use.lock acquired by nodename 14467@lucene1-us-west
   [junit4]   2> 6802 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:33481) [    ] o.a.h.h.s.c.Storage Storage directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data2 is not formatted for BP-1156588151-10.41.0.5-1447923616226
   [junit4]   2> 6802 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:33481) [    ] o.a.h.h.s.c.Storage Formatting ...
   [junit4]   2> 6802 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:33481) [    ] o.a.h.h.s.c.Storage Lock on /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data3/in_use.lock acquired by nodename 14467@lucene1-us-west
   [junit4]   2> 6803 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:33481) [    ] o.a.h.h.s.c.Storage Storage directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data3 is not formatted for BP-1156588151-10.41.0.5-1447923616226
   [junit4]   2> 6803 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:33481) [    ] o.a.h.h.s.c.Storage Formatting ...
   [junit4]   2> 6811 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:33481) [    ] o.a.h.h.s.c.Storage Lock on /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data4/in_use.lock acquired by nodename 14467@lucene1-us-west
   [junit4]   2> 6812 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:33481) [    ] o.a.h.h.s.c.Storage Storage directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data4 is not formatted for BP-1156588151-10.41.0.5-1447923616226
   [junit4]   2> 6812 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:33481) [    ] o.a.h.h.s.c.Storage Formatting ...
   [junit4]   2> 7021 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:33481) [    ] o.a.h.h.s.c.Storage Analyzing storage directories for bpid BP-1156588151-10.41.0.5-1447923616226
   [junit4]   2> 7021 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:33481) [    ] o.a.h.h.s.c.Storage Locking is disabled
   [junit4]   2> 7023 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:33481) [    ] o.a.h.h.s.c.Storage Storage directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data3/current/BP-1156588151-10.41.0.5-1447923616226 is not formatted.
   [junit4]   2> 7024 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:33481) [    ] o.a.h.h.s.c.Storage Formatting ...
   [junit4]   2> 7024 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:33481) [    ] o.a.h.h.s.c.Storage Formatting block pool BP-1156588151-10.41.0.5-1447923616226 directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data3/current/BP-1156588151-10.41.0.5-1447923616226/current
   [junit4]   2> 7028 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:33481) [    ] o.a.h.h.s.c.Storage Locking is disabled
   [junit4]   2> 7029 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:33481) [    ] o.a.h.h.s.c.Storage Storage directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data4/current/BP-1156588151-10.41.0.5-1447923616226 is not formatted.
   [junit4]   2> 7029 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:33481) [    ] o.a.h.h.s.c.Storage Formatting ...
   [junit4]   2> 7029 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:33481) [    ] o.a.h.h.s.c.Storage Formatting block pool BP-1156588151-10.41.0.5-1447923616226 directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data4/current/BP-1156588151-10.41.0.5-1447923616226/current
   [junit4]   2> 7042 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:33481) [    ] o.a.h.h.s.c.Storage Restored 0 block files from trash.
   [junit4]   2> 7043 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:33481) [    ] o.a.h.h.s.c.Storage Restored 0 block files from trash.
   [junit4]   2> 7044 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:33481) [    ] o.a.h.h.s.c.Storage Analyzing storage directories for bpid BP-1156588151-10.41.0.5-1447923616226
   [junit4]   2> 7045 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:33481) [    ] o.a.h.h.s.c.Storage Locking is disabled
   [junit4]   2> 7046 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:33481) [    ] o.a.h.h.s.c.Storage Storage directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data1/current/BP-1156588151-10.41.0.5-1447923616226 is not formatted.
   [junit4]   2> 7046 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:33481) [    ] o.a.h.h.s.c.Storage Formatting ...
   [junit4]   2> 7046 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:33481) [    ] o.a.h.h.s.c.Storage Formatting block pool BP-1156588151-10.41.0.5-1447923616226 directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data1/current/BP-1156588151-10.41.0.5-1447923616226/current
   [junit4]   2> 7050 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:33481) [    ] o.a.h.h.s.c.Storage Locking is disabled
   [junit4]   2> 7051 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:33481) [    ] o.a.h.h.s.c.Storage Storage directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data2/current/BP-1156588151-10.41.0.5-1447923616226 is not formatted.
   [junit4]   2> 7051 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:33481) [    ] o.a.h.h.s.c.Storage Formatting ...
   [junit4]   2> 7051 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:33481) [    ] o.a.h.h.s.c.Storage Formatting block pool BP-1156588151-10.41.0.5-1447923616226 directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data2/current/BP-1156588151-10.41.0.5-1447923616226/current
   [junit4]   2> 7055 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:33481) [    ] o.a.h.h.s.d.DataNode Setting up storage: nsid=2035105108;bpid=BP-1156588151-10.41.0.5-1447923616226;lv=-56;nsInfo=lv=-60;cid=testClusterID;nsid=2035105108;c=0;bpid=BP-1156588151-10.41.0.5-1447923616226;dnuuid=null
   [junit4]   2> 7056 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:33481) [    ] o.a.h.h.s.c.Storage Restored 0 block files from trash.
   [junit4]   2> 7057 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:33481) [    ] o.a.h.h.s.c.Storage Restored 0 block files from trash.
   [junit4]   2> 7060 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:33481) [    ] o.a.h.h.s.d.DataNode Setting up storage: nsid=2035105108;bpid=BP-1156588151-10.41.0.5-1447923616226;lv=-56;nsInfo=lv=-60;cid=testClusterID;nsid=2035105108;c=0;bpid=BP-1156588151-10.41.0.5-1447923616226;dnuuid=null
   [junit4]   2> 7062 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:33481) [    ] o.a.h.h.s.d.DataNode Generated and persisted new Datanode UUID 4d668143-e162-4746-bc91-d7aa9aab5633
   [junit4]   2> 7067 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:33481) [    ] o.a.h.h.s.d.DataNode Generated and persisted new Datanode UUID 9f1586ea-0d50-45d5-8e68-dd31c9f612be
   [junit4]   2> 7143 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:33481) [    ] o.a.h.h.s.d.f.i.FsDatasetImpl Added new volume: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data3/current
   [junit4]   2> 7145 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:33481) [    ] o.a.h.h.s.d.f.i.FsDatasetImpl Added volume - /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data3/current, StorageType: DISK
   [junit4]   2> 7146 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:33481) [    ] o.a.h.h.s.d.f.i.FsDatasetImpl Added new volume: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data4/current
   [junit4]   2> 7146 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:33481) [    ] o.a.h.h.s.d.f.i.FsDatasetImpl Added volume - /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data4/current, StorageType: DISK
   [junit4]   2> 7149 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:33481) [    ] o.a.h.h.s.d.f.i.FsDatasetImpl Added new volume: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data1/current
   [junit4]   2> 7152 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:33481) [    ] o.a.h.h.s.d.f.i.FsDatasetImpl Added volume - /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data1/current, StorageType: DISK
   [junit4]   2> 7156 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:33481) [    ] o.a.h.h.s.d.f.i.FsDatasetImpl Added new volume: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data2/current
   [junit4]   2> 7159 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:33481) [    ] o.a.h.h.s.d.f.i.FsDatasetImpl Added volume - /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data2/current, StorageType: DISK
   [junit4]   2> 7194 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data4/]]  heartbeating to localhost/127.0.0.1:33481) [    ] o.a.h.h.s.d.f.i.FsDatasetImpl Registered FSDatasetState MBean
   [junit4]   2> 7194 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/solr.hadoop.MorphlineBasicMiniMRTest_769F8DA6084321B3-001/tempDir-002/hdfs/build/dfs/data/data2/]]  heartbeating to localhost/127.0.0.1:33481) [    ] o.a.h.h.s.d.f.i.FsDatasetImpl Registered FSDatasetState MBean
   [junit4]   2> 7202 INFO  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J0/temp/so

[...truncated too long message...]

er) [n:127.0.0.1:42010_ c:collection1 s:shard1 r:core_node5 x:collection1] o.a.h.h.s.n.FSNamesystem Stopping services started for standby state
   [junit4]   2> 79921 INFO  (SUITE-MorphlineGoLiveMiniMRTest-seed#[769F8DA6084321B3]-worker) [n:127.0.0.1:42010_ c:collection1 s:shard1 r:core_node5 x:collection1] o.m.log Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0
   [junit4]   2> 80023 INFO  (SUITE-MorphlineGoLiveMiniMRTest-seed#[769F8DA6084321B3]-worker) [n:127.0.0.1:42010_ c:collection1 s:shard1 r:core_node5 x:collection1] o.a.h.m.i.MetricsSystemImpl Stopping JobHistoryServer metrics system...
   [junit4]   2> 80026 ERROR (SUITE-MorphlineGoLiveMiniMRTest-seed#[769F8DA6084321B3]-worker) [n:127.0.0.1:42010_ c:collection1 s:shard1 r:core_node5 x:collection1] o.a.h.m.l.MethodMetric Error invoking method getBlocksTotal
   [junit4]   2> java.lang.reflect.InvocationTargetException
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   [junit4]   2> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2> 	at java.lang.reflect.Method.invoke(Method.java:606)
   [junit4]   2> 	at org.apache.hadoop.metrics2.lib.MethodMetric$2.snapshot(MethodMetric.java:111)
   [junit4]   2> 	at org.apache.hadoop.metrics2.lib.MethodMetric.snapshot(MethodMetric.java:144)
   [junit4]   2> 	at org.apache.hadoop.metrics2.lib.MetricsRegistry.snapshot(MetricsRegistry.java:387)
   [junit4]   2> 	at org.apache.hadoop.metrics2.lib.MetricsSourceBuilder$1.getMetrics(MetricsSourceBuilder.java:79)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.getMetrics(MetricsSourceAdapter.java:195)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.updateJmxCache(MetricsSourceAdapter.java:172)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.getMBeanInfo(MetricsSourceAdapter.java:151)
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getClassName(DefaultMBeanServerInterceptor.java:1804)
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.safeGetClassName(DefaultMBeanServerInterceptor.java:1595)
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.checkMBeanPermission(DefaultMBeanServerInterceptor.java:1813)
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.exclusiveUnregisterMBean(DefaultMBeanServerInterceptor.java:430)
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.unregisterMBean(DefaultMBeanServerInterceptor.java:415)
   [junit4]   2> 	at com.sun.jmx.mbeanserver.JmxMBeanServer.unregisterMBean(JmxMBeanServer.java:546)
   [junit4]   2> 	at org.apache.hadoop.metrics2.util.MBeans.unregister(MBeans.java:81)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.stopMBeans(MetricsSourceAdapter.java:227)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.stop(MetricsSourceAdapter.java:212)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.stopSources(MetricsSystemImpl.java:461)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.stop(MetricsSystemImpl.java:212)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.shutdown(MetricsSystemImpl.java:592)
   [junit4]   2> 	at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.shutdownInstance(DefaultMetricsSystem.java:72)
   [junit4]   2> 	at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.shutdown(DefaultMetricsSystem.java:68)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.metrics.NameNodeMetrics.shutdown(NameNodeMetrics.java:145)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.NameNode.stop(NameNode.java:822)
   [junit4]   2> 	at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1720)
   [junit4]   2> 	at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1699)
   [junit4]   2> 	at org.apache.solr.hadoop.MorphlineGoLiveMiniMRTest.teardownClass(MorphlineGoLiveMiniMRTest.java:236)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   [junit4]   2> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2> 	at java.lang.reflect.Method.invoke(Method.java:606)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:834)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> Caused by: java.lang.NullPointerException
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.blockmanagement.BlocksMap.size(BlocksMap.java:198)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.getTotalBlocks(BlockManager.java:3291)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlocksTotal(FSNamesystem.java:6223)
   [junit4]   2> 	... 53 more
   [junit4]   2> 80040 INFO  (SUITE-MorphlineGoLiveMiniMRTest-seed#[769F8DA6084321B3]-worker) [n:127.0.0.1:42010_ c:collection1 s:shard1 r:core_node5 x:collection1] o.a.h.m.i.MetricsSystemImpl JobHistoryServer metrics system stopped.
   [junit4]   2> 80041 INFO  (SUITE-MorphlineGoLiveMiniMRTest-seed#[769F8DA6084321B3]-worker) [n:127.0.0.1:42010_ c:collection1 s:shard1 r:core_node5 x:collection1] o.a.h.m.i.MetricsSystemImpl JobHistoryServer metrics system shutdown complete.
   [junit4]   2> 80042 INFO  (SUITE-MorphlineGoLiveMiniMRTest-seed#[769F8DA6084321B3]-worker) [n:127.0.0.1:42010_ c:collection1 s:shard1 r:core_node5 x:collection1] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-map-reduce/test/J1/temp/solr.hadoop.MorphlineGoLiveMiniMRTest_769F8DA6084321B3-001
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene54): {}, docValues:{}, sim=RandomSimilarityProvider(queryNorm=false,coord=crazy): {}, locale=no, timezone=Indian/Comoro
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_80 (64-bit)/cpus=4,threads=9,free=216583448,total=337641472
   [junit4]   2> NOTE: All tests run in this JVM: [MorphlineGoLiveMiniMRTest]
   [junit4] Completed [7/7] on J1 in 81.69s, 2 tests, 1 error <<< FAILURES!

[...truncated 12 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:253: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:467: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 7 suites, 35 tests, 2 errors, 1 ignored [seed: 769F8DA6084321B3]

Total time: 208 minutes 8 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1020 - Still Failing

[Apache Jenkins Server <je...@builds.apache.org>]

Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1020/

1 tests failed.
FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Could not load collection from ZK:nodes_used_collection

Stack Trace:
org.apache.solr.common.SolrException: Could not load collection from ZK:nodes_used_collection
	at org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:1022)
	at org.apache.solr.common.cloud.ZkStateReader$LazyCollectionRef.get(ZkStateReader.java:555)
	at org.apache.solr.common.cloud.ClusterState.getCollections(ClusterState.java:207)
	at org.apache.solr.common.cloud.ZkStateReader.constructState(ZkStateReader.java:461)
	at org.apache.solr.common.cloud.ZkStateReader.updateClusterState(ZkStateReader.java:266)
	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:144)
	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:131)
	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:126)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.waitForRecoveriesToFinish(AbstractFullDistribZkTestBase.java:830)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.deleteCollectionWithDownNodes(CollectionsAPIDistributedZkTest.java:251)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test(CollectionsAPIDistributedZkTest.java:163)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1660)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:866)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:902)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:916)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:963)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:938)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:875)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:777)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:811)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:822)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/nodes_used_collection/state.json
	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
	at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
	at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:345)
	at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:342)
	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:61)
	at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:342)
	at org.apache.solr.common.cloud.ZkStateReader.fetchCollectionState(ZkStateReader.java:1035)
	at org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:1020)
	... 51 more




Build Log:
[...truncated 11024 lines...]
   [junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D89271C806E95B6A-001/init-core-data-001
   [junit4]   2> 990323 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[D89271C806E95B6A]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false)
   [junit4]   2> 990323 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[D89271C806E95B6A]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   2> 990328 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 990329 INFO  (Thread-6443) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 990329 INFO  (Thread-6443) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 990429 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.ZkTestServer start zk server on port:39182
   [junit4]   2> 990429 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 990429 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 990449 INFO  (zkCallback-2385-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5061d28 name:ZooKeeperConnection Watcher:127.0.0.1:39182 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 990449 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 990449 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 990450 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 990453 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 990454 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 990455 INFO  (zkCallback-2386-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@8a6ef51 name:ZooKeeperConnection Watcher:127.0.0.1:39182/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 990455 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 990455 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 990455 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 990457 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 990459 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 990460 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 990463 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 990463 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 990467 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 990467 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 990469 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 990469 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 990471 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 990471 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 990473 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 990473 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 990476 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 990476 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 990478 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 990478 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 990481 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 990481 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 990483 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 990483 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 990485 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 990485 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 990487 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 990487 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 990489 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 990490 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 990492 INFO  (zkCallback-2387-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@79bc1b05 name:ZooKeeperConnection Watcher:127.0.0.1:39182 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 990492 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 990492 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 990492 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 990494 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 990495 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 990496 INFO  (zkCallback-2388-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@14be6b6c name:ZooKeeperConnection Watcher:127.0.0.1:39182/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 990496 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 990496 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 990497 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf2/solrconfig.xml
   [junit4]   2> 990497 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/solrconfig.xml
   [junit4]   2> 990500 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf2/schema.xml
   [junit4]   2> 990500 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/schema.xml
   [junit4]   2> 990502 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf2/enumsConfig.xml
   [junit4]   2> 990502 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/enumsConfig.xml
   [junit4]   2> 990504 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf2/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 990504 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 990506 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf2/stopwords.txt
   [junit4]   2> 990506 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/stopwords.txt
   [junit4]   2> 990511 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf2/protwords.txt
   [junit4]   2> 990511 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/protwords.txt
   [junit4]   2> 990515 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf2/currency.xml
   [junit4]   2> 990515 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/currency.xml
   [junit4]   2> 990517 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf2/open-exchange-rates.json
   [junit4]   2> 990517 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/open-exchange-rates.json
   [junit4]   2> 990519 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf2/mapping-ISOLatin1Accent.txt
   [junit4]   2> 990519 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/mapping-ISOLatin1Accent.txt
   [junit4]   2> 990522 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf2/old_synonyms.txt
   [junit4]   2> 990522 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/old_synonyms.txt
   [junit4]   2> 990524 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf2/synonyms.txt
   [junit4]   2> 990524 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/synonyms.txt
   [junit4]   2> 990528 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/elevate.xml to /configs/conf2/elevate.xml
   [junit4]   2> 990528 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/elevate.xml
   [junit4]   2> 990814 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D89271C806E95B6A-001/control-001/cores/collection1
   [junit4]   2> 990816 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 990822 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5e100cdd{/,null,AVAILABLE}
   [junit4]   2> 990824 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.e.j.s.ServerConnector Started ServerConnector@4d239dce{HTTP/1.1}{127.0.0.1:54357}
   [junit4]   2> 990824 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.e.j.s.Server Started @993717ms
   [junit4]   2> 990824 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostPort=54357, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D89271C806E95B6A-001/tempDir-001/control/data, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D89271C806E95B6A-001/control-001/cores, hostContext=/}
   [junit4]   2> 990824 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@7b3cb2c6
   [junit4]   2> 990824 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D89271C806E95B6A-001/control-001/'
   [junit4]   2> 990825 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 990825 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 990849 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 990849 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 990853 INFO  (zkCallback-2389-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5a7ffa2a name:ZooKeeperConnection Watcher:127.0.0.1:39182/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 990853 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 990853 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 990854 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 990854 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D89271C806E95B6A-001/control-001/solr.xml
   [junit4]   2> 990868 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D89271C806E95B6A-001/control-001/cores
   [junit4]   2> 990868 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.CoreContainer New CoreContainer 981056402
   [junit4]   2> 990868 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D89271C806E95B6A-001/control-001/]
   [junit4]   2> 990868 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D89271C806E95B6A-001/control-001/lib
   [junit4]   2> 990869 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D89271C806E95B6A-001/control-001/lib).
   [junit4]   2> 990879 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 990880 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 990881 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 990881 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 990881 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:39182/solr
   [junit4]   2> 990881 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 990881 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 990882 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 990883 INFO  (zkCallback-2391-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@792da9de name:ZooKeeperConnection Watcher:127.0.0.1:39182 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 990884 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 990884 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 990888 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [n:127.0.0.1:54357_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 990889 INFO  (zkCallback-2392-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@30795bfc name:ZooKeeperConnection Watcher:127.0.0.1:39182/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 990889 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [n:127.0.0.1:54357_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 990892 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [n:127.0.0.1:54357_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 990894 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [n:127.0.0.1:54357_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 990898 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [n:127.0.0.1:54357_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 990901 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [n:127.0.0.1:54357_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 990903 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [n:127.0.0.1:54357_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 990905 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [n:127.0.0.1:54357_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 990906 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [n:127.0.0.1:54357_    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 990907 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [n:127.0.0.1:54357_    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 990908 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [n:127.0.0.1:54357_    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 990909 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [n:127.0.0.1:54357_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:54357_
   [junit4]   2> 990909 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [n:127.0.0.1:54357_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:54357_
   [junit4]   2> 990917 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [n:127.0.0.1:54357_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 990918 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [n:127.0.0.1:54357_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 990919 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [n:127.0.0.1:54357_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 990920 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [n:127.0.0.1:54357_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94883104232767494-127.0.0.1:54357_-n_0000000000
   [junit4]   2> 990921 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [n:127.0.0.1:54357_    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:54357_
   [junit4]   2> 990921 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [n:127.0.0.1:54357_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 990923 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [n:127.0.0.1:54357_    ] o.a.s.c.Overseer Overseer (id=94883104232767494-127.0.0.1:54357_-n_0000000000) starting
   [junit4]   2> 990924 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [n:127.0.0.1:54357_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 990932 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [n:127.0.0.1:54357_    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 990932 INFO  (OverseerCollectionConfigSetProcessor-94883104232767494-127.0.0.1:54357_-n_0000000000) [n:127.0.0.1:54357_    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 990933 INFO  (OverseerStateUpdate-94883104232767494-127.0.0.1:54357_-n_0000000000) [n:127.0.0.1:54357_    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 990933 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [n:127.0.0.1:54357_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 990953 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [n:127.0.0.1:54357_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 990954 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [n:127.0.0.1:54357_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 990955 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [n:127.0.0.1:54357_    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D89271C806E95B6A-001/control-001/cores
   [junit4]   2> 990956 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [n:127.0.0.1:54357_    ] o.a.s.c.SolrCore Created CoreDescriptor: {absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D89271C806E95B6A-001/control-001/cores/collection1/, transient=false, schema=schema.xml, configSetProperties=configsetprops.json, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D89271C806E95B6A-001/control-001/cores/collection1, loadOnStartup=true, dataDir=data/, shard=, name=collection1, coreNodeName=, config=solrconfig.xml, collection=control_collection}
   [junit4]   2> 990956 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [n:127.0.0.1:54357_    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D89271C806E95B6A-001/control-001/cores/collection1/
   [junit4]   2> 990956 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [n:127.0.0.1:54357_    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 990957 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [n:127.0.0.1:54357_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 990957 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [n:127.0.0.1:54357_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 990958 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 990958 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 990959 INFO  (OverseerStateUpdate-94883104232767494-127.0.0.1:54357_-n_0000000000) [n:127.0.0.1:54357_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:54357",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "node_name":"127.0.0.1:54357_",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "state":"down"} current state version: 0
   [junit4]   2> 990960 INFO  (OverseerStateUpdate-94883104232767494-127.0.0.1:54357_-n_0000000000) [n:127.0.0.1:54357_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:54357",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "node_name":"127.0.0.1:54357_",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "state":"down"}
   [junit4]   2> 990960 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 990961 INFO  (OverseerStateUpdate-94883104232767494-127.0.0.1:54357_-n_0000000000) [n:127.0.0.1:54357_    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 990961 INFO  (OverseerStateUpdate-94883104232767494-127.0.0.1:54357_-n_0000000000) [n:127.0.0.1:54357_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 990963 INFO  (zkCallback-2392-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 991961 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 991961 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 991963 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 991963 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 991964 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 991964 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D89271C806E95B6A-001/control-001/cores/collection1/'
   [junit4]   2> 991964 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 991964 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 991984 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 991989 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 992004 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 992034 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 992036 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 992042 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 992317 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 992324 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 992326 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 992336 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 992339 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 992342 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 992343 WARN  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 992343 WARN  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 992343 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 992344 WARN  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 992344 WARN  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 992345 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 992345 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 992345 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 992346 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D89271C806E95B6A-001/control-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 992346 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@20603147
   [junit4]   2> 992346 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D89271C806E95B6A-001/control-001/cores/collection1/data
   [junit4]   2> 992346 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D89271C806E95B6A-001/control-001/cores/collection1/data/index/
   [junit4]   2> 992346 WARN  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D89271C806E95B6A-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 992346 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D89271C806E95B6A-001/control-001/cores/collection1/data/index
   [junit4]   2> 992347 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=42, maxMergeAtOnceExplicit=29, maxMergedSegmentMB=43.2099609375, floorSegmentMB=1.296875, forceMergeDeletesPctAllowed=8.243449821918947, segmentsPerTier=44.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 992347 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(NRTCachingDirectory(RAMDirectory@5a70997a lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@166c46f9; maxCacheMB=0.23046875 maxMergeSizeMB=0.6484375)),segFN=segments_1,generation=1}
   [junit4]   2> 992347 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 992348 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D89271C806E95B6A-001/control-001/cores/collection1/data/
   [junit4]   2> 992348 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D89271C806E95B6A-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 992352 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 992352 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 992353 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 992353 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 992353 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 992353 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 992353 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 992353 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 992353 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 992354 WARN  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
   [junit4]   2> 992358 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 992359 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 992361 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 992362 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 992367 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: standard,/get,/admin/mbeans,/replication,/admin/segments,/admin/properties,/update/json,/update,/admin/ping,/admin/plugins,/update/json/docs,/update/csv,/admin/system,/dump,/config,/admin/luke,/admin/file,/admin/logging,/admin/threads,/schema
   [junit4]   2> 992369 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 992369 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 992369 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 992370 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 992370 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 992370 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=49, maxMergeAtOnceExplicit=28, maxMergedSegmentMB=13.8505859375, floorSegmentMB=1.193359375, forceMergeDeletesPctAllowed=16.90147541995426, segmentsPerTier=2.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.21096095258937622
   [junit4]   2> 992371 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(NRTCachingDirectory(RAMDirectory@5a70997a lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@166c46f9; maxCacheMB=0.23046875 maxMergeSizeMB=0.6484375)),segFN=segments_1,generation=1}
   [junit4]   2> 992371 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 992371 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@35ee41ef[collection1] main
   [junit4]   2> 992371 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 992372 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 992372 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 992373 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 992373 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 992373 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 992373 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 992373 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 992374 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 992374 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 992374 INFO  (searcherExecutor-4227-thread-1-processing-r:core_node1 x:collection1 c:control_collection s:shard1 n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@35ee41ef[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 992374 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 992374 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 992374 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 992374 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1518129669867569152
   [junit4]   2> 992376 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 2.0ms to seed version buckets with highest version 1518129669867569152
   [junit4]   2> 992377 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 992378 INFO  (coreLoadExecutor-4226-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 992378 INFO  (coreZkRegister-4220-thread-1-processing-r:core_node1 x:collection1 c:control_collection s:shard1 n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:54357 collection:control_collection shard:shard1
   [junit4]   2> 992380 INFO  (coreZkRegister-4220-thread-1-processing-r:core_node1 x:collection1 c:control_collection s:shard1 n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 992385 INFO  (coreZkRegister-4220-thread-1-processing-r:core_node1 x:collection1 c:control_collection s:shard1 n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94883104232767494-core_node1-n_0000000000
   [junit4]   2> 992386 INFO  (coreZkRegister-4220-thread-1-processing-r:core_node1 x:collection1 c:control_collection s:shard1 n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 992388 INFO  (coreZkRegister-4220-thread-1-processing-r:core_node1 x:collection1 c:control_collection s:shard1 n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 992388 INFO  (coreZkRegister-4220-thread-1-processing-r:core_node1 x:collection1 c:control_collection s:shard1 n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 992388 INFO  (OverseerStateUpdate-94883104232767494-127.0.0.1:54357_-n_0000000000) [n:127.0.0.1:54357_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 992388 INFO  (coreZkRegister-4220-thread-1-processing-r:core_node1 x:collection1 c:control_collection s:shard1 n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:54357/collection1/
   [junit4]   2> 992389 INFO  (coreZkRegister-4220-thread-1-processing-r:core_node1 x:collection1 c:control_collection s:shard1 n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 992389 INFO  (coreZkRegister-4220-thread-1-processing-r:core_node1 x:collection1 c:control_collection s:shard1 n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:54357/collection1/ has no replicas
   [junit4]   2> 992390 INFO  (coreZkRegister-4220-thread-1-processing-r:core_node1 x:collection1 c:control_collection s:shard1 n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 992394 INFO  (coreZkRegister-4220-thread-1-processing-r:core_node1 x:collection1 c:control_collection s:shard1 n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 992396 INFO  (coreZkRegister-4220-thread-1-processing-r:core_node1 x:collection1 c:control_collection s:shard1 n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:54357/collection1/ shard1
   [junit4]   2> 992399 INFO  (OverseerStateUpdate-94883104232767494-127.0.0.1:54357_-n_0000000000) [n:127.0.0.1:54357_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:54357",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 1
   [junit4]   2> 992459 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 992460 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 992461 INFO  (zkCallback-2394-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@39b43132 name:ZooKeeperConnection Watcher:127.0.0.1:39182/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 992461 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 992462 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 992462 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 992464 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 992465 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with stateFormat=2
   [junit4]   2> 992465 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 992465 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 992467 INFO  (zkCallback-2395-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@242ee67f name:ZooKeeperConnection Watcher:127.0.0.1:39182/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 992467 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 992467 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 992472 INFO  (OverseerStateUpdate-94883104232767494-127.0.0.1:54357_-n_0000000000) [n:127.0.0.1:54357_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"create",
   [junit4]   2>   "name":"collection1",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "stateFormat":"2"} current state version: 1
   [junit4]   2> 992472 INFO  (OverseerStateUpdate-94883104232767494-127.0.0.1:54357_-n_0000000000) [n:127.0.0.1:54357_    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 992472 INFO  (zkCallback-2392-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 992473 INFO  (zkCallback-2394-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 992500 INFO  (coreZkRegister-4220-thread-1-processing-r:core_node1 x:collection1 c:control_collection s:shard1 n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:54357/collection1/ and leader is http://127.0.0.1:54357/collection1/
   [junit4]   2> 992500 INFO  (coreZkRegister-4220-thread-1-processing-r:core_node1 x:collection1 c:control_collection s:shard1 n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:54357
   [junit4]   2> 992500 INFO  (coreZkRegister-4220-thread-1-processing-r:core_node1 x:collection1 c:control_collection s:shard1 n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 992500 INFO  (coreZkRegister-4220-thread-1-processing-r:core_node1 x:collection1 c:control_collection s:shard1 n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 992500 INFO  (coreZkRegister-4220-thread-1-processing-r:core_node1 x:collection1 c:control_collection s:shard1 n:127.0.0.1:54357_) [n:127.0.0.1:54357_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 992502 INFO  (OverseerStateUpdate-94883104232767494-127.0.0.1:54357_-n_0000000000) [n:127.0.0.1:54357_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:54357",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "node_name":"127.0.0.1:54357_",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "state":"active",
   [junit4]   2>   "core_node_name":"core_node1"} current state version: 2
   [junit4]   2> 992502 INFO  (OverseerStateUpdate-94883104232767494-127.0.0.1:54357_-n_0000000000) [n:127.0.0.1:54357_    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:54357",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "node_name":"127.0.0.1:54357_",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "state":"active",
   [junit4]   2>   "core_node_name":"core_node1"}
   [junit4]   2> 992503 INFO  (OverseerStateUpdate-94883104232767494-127.0.0.1:54357_-n_0000000000) [n:127.0.0.1:54357_    ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/collection1/state.json
   [junit4]   2> 992609 INFO  (zkCallback-2392-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 992609 INFO  (zkCallback-2394-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 992809 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D89271C806E95B6A-001/shard-1-001/cores/collection1
   [junit4]   2> 992809 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D89271C806E95B6A-001/shard-1-001
   [junit4]   2> 992810 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 992813 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@904dd0e{/,null,AVAILABLE}
   [junit4]   2> 992813 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.e.j.s.ServerConnector Started ServerConnector@428cfa18{HTTP/1.1}{127.0.0.1:57384}
   [junit4]   2> 992813 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.e.j.s.Server Started @995706ms
   [junit4]   2> 992813 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D89271C806E95B6A-001/tempDir-001/jetty1, hostContext=/, hostPort=57384, solrconfig=solrconfig.xml, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D89271C806E95B6A-001/shard-1-001/cores}
   [junit4]   2> 992814 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@7b3cb2c6
   [junit4]   2> 992814 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D89271C806E95B6A-001/shard-1-001/'
   [junit4]   2> 992814 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 992814 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 992832 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 992832 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 992834 INFO  (zkCallback-2396-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5137cbe6 name:ZooKeeperConnection Watcher:127.0.0.1:39182/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 992835 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 992835 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 992836 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 992836 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D89271C806E95B6A-001/shard-1-001/solr.xml
   [junit4]   2> 992850 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D89271C806E95B6A-001/shard-1-001/cores
   [junit4]   2> 992850 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.CoreContainer New CoreContainer 1156068566
   [junit4]   2> 992850 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D89271C806E95B6A-001/shard-1-001/]
   [junit4]   2> 992850 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D89271C806E95B6A-001/shard-1-001/lib
   [junit4]   2> 992851 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D89271C806E95B6A-001/shard-1-001/lib).
   [junit4]   2> 992859 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 992860 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 992860 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 992861 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 992861 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:39182/solr
   [junit4]   2> 992861 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 992861 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 992861 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 992884 INFO  (zkCallback-2398-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@34dd3a18 name:ZooKeeperConnection Watcher:127.0.0.1:39182 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 992886 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 992887 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 992888 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [n:127.0.0.1:57384_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 992889 INFO  (zkCallback-2399-thread-1-processing-n:127.0.0.1:57384_) [n:127.0.0.1:57384_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@4ff3b079 name:ZooKeeperConnection Watcher:127.0.0.1:39182/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 992890 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [n:127.0.0.1:57384_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 992897 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [n:127.0.0.1:57384_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 993905 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [n:127.0.0.1:57384_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:57384_
   [junit4]   2> 993905 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [n:127.0.0.1:57384_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:57384_
   [junit4]   2> 993906 INFO  (zkCallback-2394-thread-1) [    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 993906 INFO  (zkCallback-2392-thread-1-processing-n:127.0.0.1:54357_) [n:127.0.0.1:54357_    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 993906 INFO  (zkCallback-2399-thread-1-processing-n:127.0.0.1:57384_) [n:127.0.0.1:57384_    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 993907 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [n:127.0.0.1:57384_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 993909 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [n:127.0.0.1:57384_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94883104232767499-127.0.0.1:57384_-n_0000000001
   [junit4]   2> 993910 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [n:127.0.0.1:57384_    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/94883104232767494-127.0.0.1:54357_-n_0000000000 to know if I could be the leader
   [junit4]   2> 993923 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [n:127.0.0.1:57384_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 993923 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [n:127.0.0.1:57384_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 993925 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [n:127.0.0.1:57384_    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D89271C806E95B6A-001/shard-1-001/cores
   [junit4]   2> 993925 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [n:127.0.0.1:57384_    ] o.a.s.c.SolrCore Created CoreDescriptor: {coreNodeName=, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D89271C806E95B6A-001/shard-1-001/cores/collection1, shard=, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D89271C806E95B6A-001/shard-1-001/cores/collection1/, name=collection1, collection=collection1, config=solrconfig.xml, dataDir=data/, configSetProperties=configsetprops.json, transient=false, schema=schema.xml, loadOnStartup=true}
   [junit4]   2> 993926 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[D89271C806E95B6A]) [n:127.0.0.1:57384_    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Luc

[...truncated too long message...]

220)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1573)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1594)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:876)Throwable #6: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=14230, name=collection3, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    > Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:47944: collection already exists: awholynewstresscollection_collection3_0
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([D89271C806E95B6A]:0)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1573)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1594)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:876)Throwable #7: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=14231, name=collection4, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    > Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:47944: collection already exists: awholynewstresscollection_collection4_0
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([D89271C806E95B6A]:0)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1573)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1594)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:876)
   [junit4]   2> 1996765 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[D89271C806E95B6A]-worker) [n:127.0.0.1:44846_ c:halfdeletedcollection2 s:shard2 r:core_node3 x:halfdeletedcollection2_shard2_replica3] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_D89271C806E95B6A-001
   [junit4]   2> Nov 17, 2015 11:17:59 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene54): {multiDefault=PostingsFormat(name=MockRandom), range_facet_l=PostingsFormat(name=Memory doPackFST= true), timestamp=PostingsFormat(name=Memory doPackFST= true), intDefault=PostingsFormat(name=Memory doPackFST= true), text=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), range_facet_l_dv=FSTOrd50, _version_=PostingsFormat(name=Memory doPackFST= true), id=FSTOrd50, other_tl1=PostingsFormat(name=Memory doPackFST= true), range_facet_i_dv=PostingsFormat(name=Memory doPackFST= true), a_i1=PostingsFormat(name=Memory doPackFST= true), a_t=PostingsFormat(name=Memory doPackFST= true)}, docValues:{timestamp=DocValuesFormat(name=Direct), range_facet_l_dv=DocValuesFormat(name=Lucene54), range_facet_i_dv=DocValuesFormat(name=Direct)}, sim=DefaultSimilarity, locale=es_EC, timezone=America/Mendoza
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_80 (64-bit)/cpus=4,threads=1,free=186286808,total=489160704
   [junit4]   2> NOTE: All tests run in this JVM: [TestDistributedGrouping, TestFileDictionaryLookup, SchemaVersionSpecificBehaviorTest, TestDynamicFieldResource, SolrIndexConfigTest, IgnoreCommitOptimizeUpdateProcessorFactoryTest, AnalyticsQueryTest, TestMinMaxOnMultiValuedField, BitVectorTest, TestQueryUtils, TestSolrDeletionPolicy2, BasicAuthIntegrationTest, CSVRequestHandlerTest, TestAnalyzeInfixSuggestions, SpellCheckCollatorTest, TestReplicationHandlerBackup, HdfsCollectionsAPIDistributedZkTest, TestSolrConfigHandlerCloud, UpdateRequestProcessorFactoryTest, TestLMJelinekMercerSimilarityFactory, TestSolrCLIRunExample, FileUtilsTest, AddSchemaFieldsUpdateProcessorFactoryTest, TestStandardQParsers, TestManagedSchemaFieldTypeResource, SolrPluginUtilsTest, MinimalSchemaTest, DOMUtilTest, DistributedMLTComponentTest, EnumFieldTest, DocumentBuilderTest, AsyncCallRequestStatusResponseTest, SmileWriterTest, FacetPivotSmallTest, SpatialFilterTest, TestWordDelimiterFilterFactory, TestSimpleQParserPlugin, IndexBasedSpellCheckerTest, UniqFieldsUpdateProcessorFactoryTest, DistributedTermsComponentTest, TestFastLRUCache, TestQuerySenderListener, TestConfigSetsAPIZkFailure, TestBlobHandler, RankQueryTest, TestReRankQParserPlugin, CursorMarkTest, ChaosMonkeySafeLeaderTest, DeleteLastCustomShardedReplicaTest, BJQParserTest, TestImplicitCoreProperties, TermVectorComponentTest, TestExclusionRuleCollectionAccess, TestSchemaResource, ZkCLITest, SpellCheckComponentTest, TestRuleBasedAuthorizationPlugin, TestFieldTypeResource, TestLFUCache, BufferStoreTest, TestSchemaVersionResource, SolrInfoMBeanTest, TestJsonFacets, CoreAdminRequestStatusTest, PreAnalyzedUpdateProcessorTest, TestSolrQueryParserDefaultOperatorResource, SimpleMLTQParserTest, ClusterStateUpdateTest, MBeansHandlerTest, TestTrie, ConvertedLegacyTest, CloudMLTQParserTest, LeaderInitiatedRecoveryOnCommitTest, HighlighterTest, MoreLikeThisHandlerTest, SecurityConfHandlerTest, AssignTest, TestDFRSimilarityFactory, SimplePostToolTest, TermsComponentTest, DocValuesMissingTest, AddBlockUpdateTest, MultiTermTest, TestCloudManagedSchema, TestCustomDocTransformer, TestManagedSchemaFieldResource, TestBinaryResponseWriter, CollectionsAPIDistributedZkTest]
   [junit4] Completed [282/547] on J1 in 1006.98s, 1 test, 1 error <<< FAILURES!

[...truncated 853 lines...]
   [junit4] JVM J1: stdout was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J1-20151117_224439_749.sysout
   [junit4] >>> JVM J1: stdout (verbatim) ----
   [junit4] java.lang.OutOfMemoryError: GC overhead limit exceeded
   [junit4] Dumping heap to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/heapdumps/java_pid14910.hprof ...
   [junit4] Heap dump file created [671755793 bytes in 14.849 secs]
   [junit4] <<< JVM J1: EOF ----

[...truncated 16 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 547 suites (7 ignored), 2164 tests, 1 error, 32 ignored (18 assumptions) [seed: D89271C806E95B6A]

Total time: 219 minutes 3 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1019 - Still Failing

[Apache Jenkins Server <je...@builds.apache.org>]

Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1019/

3 tests failed.
FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
GC overhead limit exceeded

Stack Trace:
java.lang.OutOfMemoryError: GC overhead limit exceeded


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.CollectionsAPIDistributedZkTest

Error Message:
329 threads leaked from SUITE scope at org.apache.solr.cloud.CollectionsAPIDistributedZkTest:     1) Thread[id=21909, name=qtp1335420383-21909, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.common.cloud.ZkStateReader.getLeaderRetry(ZkStateReader.java:640)         at org.apache.solr.common.cloud.ZkStateReader.getLeaderUrl(ZkStateReader.java:609)         at org.apache.solr.cloud.ZkController.getLeader(ZkController.java:1004)         at org.apache.solr.cloud.ZkController.register(ZkController.java:939)         at org.apache.solr.cloud.ZkController.register(ZkController.java:882)         at org.apache.solr.core.ZkContainer$2.run(ZkContainer.java:183)         at org.apache.solr.core.ZkContainer.registerInZk(ZkContainer.java:212)         at org.apache.solr.core.CoreContainer.registerCore(CoreContainer.java:688)         at org.apache.solr.core.CoreContainer.create(CoreContainer.java:742)         at org.apache.solr.core.CoreContainer.create(CoreContainer.java:708)         at org.apache.solr.handler.admin.CoreAdminHandler.handleCreateAction(CoreAdminHandler.java:633)         at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestInternal(CoreAdminHandler.java:214)         at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:194)         at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:151)         at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:673)         at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:441)         at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:220)         at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:179)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:109)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.eclipse.jetty.servlets.UserAgentFilter.doFilter(UserAgentFilter.java:83)         at org.eclipse.jetty.servlets.GzipFilter.doFilter(GzipFilter.java:300)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)         at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:221)         at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)         at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)         at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)         at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)         at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)         at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)         at org.eclipse.jetty.server.Server.handle(Server.java:499)         at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)         at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)         at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)         at java.lang.Thread.run(Thread.java:745)    2) Thread[id=23172, name=Thread-13762, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.common.cloud.ZkCmdExecutor.retryDelay(ZkCmdExecutor.java:108)         at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:76)         at org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:291)         at org.apache.solr.core.SolrCore.checkStale(SolrCore.java:2628)         at org.apache.solr.core.SolrCore.access$300(SolrCore.java:153)         at org.apache.solr.core.SolrCore$11.run(SolrCore.java:2598)         at org.apache.solr.cloud.ZkController$5.run(ZkController.java:2478)    3) Thread[id=19970, name=qtp1335420383-19970-acceptor-0@7d2a04f0-ServerConnector@9fd449{SSL-http/1.1}{127.0.0.1:50058}, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250)         at org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:377)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:500)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)         at java.lang.Thread.run(Thread.java:745)    4) Thread[id=21886, name=qtp989002053-21886, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.common.cloud.ZkStateReader.getLeaderRetry(ZkStateReader.java:640)         at org.apache.solr.common.cloud.ZkStateReader.getLeaderUrl(ZkStateReader.java:609)         at org.apache.solr.cloud.ZkController.getLeader(ZkController.java:1004)         at org.apache.solr.cloud.ZkController.register(ZkController.java:939)         at org.apache.solr.cloud.ZkController.register(ZkController.java:882)         at org.apache.solr.core.ZkContainer$2.run(ZkContainer.java:183)         at org.apache.solr.core.ZkContainer.registerInZk(ZkContainer.java:212)         at org.apache.solr.core.CoreContainer.registerCore(CoreContainer.java:688)         at org.apache.solr.core.CoreContainer.create(CoreContainer.java:742)         at org.apache.solr.core.CoreContainer.create(CoreContainer.java:708)         at org.apache.solr.handler.admin.CoreAdminHandler.handleCreateAction(CoreAdminHandler.java:633)         at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestInternal(CoreAdminHandler.java:214)         at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:194)         at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:151)         at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:673)         at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:441)         at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:220)         at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:179)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:109)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.eclipse.jetty.servlets.UserAgentFilter.doFilter(UserAgentFilter.java:83)         at org.eclipse.jetty.servlets.GzipFilter.doFilter(GzipFilter.java:300)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)         at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:221)         at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)         at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)         at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)         at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)         at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)         at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)         at org.eclipse.jetty.server.Server.handle(Server.java:499)         at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)         at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)         at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)         at java.lang.Thread.run(Thread.java:745)    5) Thread[id=22975, name=Thread-13608, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.common.cloud.ZkCmdExecutor.retryDelay(ZkCmdExecutor.java:108)         at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:76)         at org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:291)         at org.apache.solr.core.RequestParams.getFreshRequestParams(RequestParams.java:148)         at org.apache.solr.core.SolrConfig.refreshRequestParams(SolrConfig.java:931)         at org.apache.solr.core.SolrCore$11.run(SolrCore.java:2596)         at org.apache.solr.cloud.ZkController$5.run(ZkController.java:2478)    6) Thread[id=20225, name=searcherExecutor-3585-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)    7) Thread[id=21568, name=qtp1335420383-21568, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.common.cloud.ZkStateReader.getLeaderRetry(ZkStateReader.java:640)         at org.apache.solr.common.cloud.ZkStateReader.getLeaderUrl(ZkStateReader.java:609)         at org.apache.solr.cloud.ZkController.getLeader(ZkController.java:1004)         at org.apache.solr.cloud.ZkController.register(ZkController.java:939)         at org.apache.solr.cloud.ZkController.register(ZkController.java:882)         at org.apache.solr.core.ZkContainer$2.run(ZkContainer.java:183)         at org.apache.solr.core.ZkContainer.registerInZk(ZkContainer.java:212)         at org.apache.solr.core.CoreContainer.registerCore(CoreContainer.java:688)         at org.apache.solr.core.CoreContainer.create(CoreContainer.java:742)         at org.apache.solr.core.CoreContainer.create(CoreContainer.java:708)         at org.apache.solr.handler.admin.CoreAdminHandler.handleCreateAction(CoreAdminHandler.java:633)         at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestInternal(CoreAdminHandler.java:214)         at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:194)         at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:151)         at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:673)         at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:441)         at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:220)         at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:179)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:109)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.eclipse.jetty.servlets.UserAgentFilter.doFilter(UserAgentFilter.java:83)         at org.eclipse.jetty.servlets.GzipFilter.doFilter(GzipFilter.java:300)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)         at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:221)         at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)         at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)         at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)         at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)         at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)         at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)         at org.eclipse.jetty.server.Server.handle(Server.java:499)         at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)         at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)         at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)         at java.lang.Thread.run(Thread.java:745)    8) Thread[id=19937, name=searcherExecutor-3380-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)    9) Thread[id=22169, name=qtp1913862660-22169, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)         at java.lang.Thread.run(Thread.java:745)   10) Thread[id=20460, name=searcherExecutor-3693-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   11) Thread[id=20147, name=searcherExecutor-3512-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   12) Thread[id=19950, name=org.eclipse.jetty.server.session.HashSessionManager@5c6d0e69Timer, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1090)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   13) Thread[id=23155, name=zkCallback-2466-thread-30, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   14) Thread[id=20334, name=searcherExecutor-3635-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   15) Thread[id=20894, name=qtp322858882-20894, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.common.cloud.ZkStateReader.getLeaderRetry(ZkStateReader.java:640)         at org.apache.solr.common.cloud.ZkStateReader.getLeaderUrl(ZkStateReader.java:609)         at org.apache.solr.cloud.ZkController.getLeader(ZkController.java:1004)         at org.apache.solr.cloud.ZkController.register(ZkController.java:939)         at org.apache.solr.cloud.ZkController.register(ZkController.java:882)         at org.apache.solr.core.ZkContainer$2.run(ZkContainer.java:183)         at org.apache.solr.core.ZkContainer.registerInZk(ZkContainer.java:212)         at org.apache.solr.core.CoreContainer.registerCore(CoreContainer.java:688)         at org.apache.solr.core.CoreContainer.create(CoreContainer.java:742)         at org.apache.solr.core.CoreContainer.create(CoreContainer.java:708)         at org.apache.solr.handler.admin.CoreAdminHandler.handleCreateAction(CoreAdminHandler.java:633)         at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestInternal(CoreAdminHandler.java:214)         at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:194)         at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:151)         at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:673)         at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:441)         at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:220)         at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:179)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:109)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.eclipse.jetty.servlets.UserAgentFilter.doFilter(UserAgentFilter.java:83)         at org.eclipse.jetty.servlets.GzipFilter.doFilter(GzipFilter.java:300)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)         at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:221)         at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)         at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)         at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)         at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)         at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)         at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)         at org.eclipse.jetty.server.Server.handle(Server.java:499)         at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)         at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)         at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)         at java.lang.Thread.run(Thread.java:745)   16) Thread[id=20350, name=searcherExecutor-3649-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   17) Thread[id=21916, name=qtp1913862660-21916, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.common.cloud.ZkStateReader.getLeaderRetry(ZkStateReader.java:640)         at org.apache.solr.common.cloud.ZkStateReader.getLeaderUrl(ZkStateReader.java:609)         at org.apache.solr.cloud.ZkController.getLeader(ZkController.java:1004)         at org.apache.solr.cloud.ZkController.register(ZkController.java:939)         at org.apache.solr.cloud.ZkController.register(ZkController.java:882)         at org.apache.solr.core.ZkContainer$2.run(ZkContainer.java:183)         at org.apache.solr.core.ZkContainer.registerInZk(ZkContainer.java:212)         at org.apache.solr.core.CoreContainer.registerCore(CoreContainer.java:688)         at org.apache.solr.core.CoreContainer.create(CoreContainer.java:742)         at org.apache.solr.core.CoreContainer.create(CoreContainer.java:708)         at org.apache.solr.handler.admin.CoreAdminHandler.handleCreateAction(CoreAdminHandler.java:633)         at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestInternal(CoreAdminHandler.java:214)         at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:194)         at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:151)         at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:673)         at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:441)         at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:220)         at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:179)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:109)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.eclipse.jetty.servlets.UserAgentFilter.doFilter(UserAgentFilter.java:83)         at org.eclipse.jetty.servlets.GzipFilter.doFilter(GzipFilter.java:300)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)         at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:221)         at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)         at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)         at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)         at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)         at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)         at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)         at org.eclipse.jetty.server.Server.handle(Server.java:499)         at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)         at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)         at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)         at java.lang.Thread.run(Thread.java:745)   18) Thread[id=23203, name=zkCallback-2505-thread-60, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   19) Thread[id=19998, name=Scheduler-1279661854, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1090)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   20) Thread[id=22194, name=qtp1335420383-22194, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.common.cloud.ZkCmdExecutor.retryDelay(ZkCmdExecutor.java:108)         at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:76)         at org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:308)         at org.apache.solr.cloud.ZkController.pathExists(ZkController.java:872)         at org.apache.solr.cloud.ZkSolrResourceLoader.openResource(ZkSolrResourceLoader.java:82)         at org.apache.solr.core.SolrResourceLoader.openSchema(SolrResourceLoader.java:309)         at org.apache.solr.schema.IndexSchemaFactory.create(IndexSchemaFactory.java:47)         at org.apache.solr.schema.IndexSchemaFactory.buildIndexSchema(IndexSchemaFactory.java:69)         at org.apache.solr.core.ConfigSetService.createIndexSchema(ConfigSetService.java:107)         at org.apache.solr.core.ConfigSetService.getConfig(ConfigSetService.java:78)         at org.apache.solr.core.CoreContainer.create(CoreContainer.java:732)         at org.apache.solr.core.CoreContainer.create(CoreContainer.java:708)         at org.apache.solr.handler.admin.CoreAdminHandler.handleCreateAction(CoreAdminHandler.java:633)         at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestInternal(CoreAdminHandler.java:214)         at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:194)         at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:151)         at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:673)         at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:441)         at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:220)         at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:179)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:109)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.eclipse.jetty.servlets.UserAgentFilter.doFilter(UserAgentFilter.java:83)         at org.eclipse.jetty.servlets.GzipFilter.doFilter(GzipFilter.java:300)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)         at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:221)         at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)         at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)         at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)         at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)         at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)         at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)         at org.eclipse.jetty.server.Server.handle(Server.java:499)         at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)         at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)         at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)         at java.lang.Thread.run(Thread.java:745)   21) Thread[id=21881, name=qtp1335420383-21881, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.common.cloud.ZkStateReader.getLeaderRetry(ZkStateReader.java:640)         at org.apache.solr.common.cloud.ZkStateReader.getLeaderUrl(ZkStateReader.java:609)         at org.apache.solr.cloud.ZkController.getLeader(ZkController.java:1004)         at org.apache.solr.cloud.ZkController.register(ZkController.java:939)         at org.apache.solr.cloud.ZkController.register(ZkController.java:882)         at org.apache.solr.core.ZkContainer$2.run(ZkContainer.java:183)         at org.apache.solr.core.ZkContainer.registerInZk(ZkContainer.java:212)         at org.apache.solr.core.CoreContainer.registerCore(CoreContainer.java:688)         at org.apache.solr.core.CoreContainer.create(CoreContainer.java:742)         at org.apache.solr.core.CoreContainer.create(CoreContainer.java:708)         at org.apache.solr.handler.admin.CoreAdminHandler.handleCreateAction(CoreAdminHandler.java:633)         at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestInternal(CoreAdminHandler.java:214)         at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:194)         at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:151)         at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:673)         at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:441)         at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:220)         at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:179)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:109)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.eclipse.jetty.servlets.UserAgentFilter.doFilter(UserAgentFilter.java:83)         at org.eclipse.jetty.servlets.GzipFilter.doFilter(GzipFilter.java:300)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)         at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:221)         at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)         at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)         at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)         at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)         at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)         at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)         at org.eclipse.jetty.server.Server.handle(Server.java:499)         at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)         at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)         at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)         at java.lang.Thread.run(Thread.java:745)   22) Thread[id=22345, name=searcherExecutor-5100-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   23) Thread[id=23058, name=Thread-13674, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.common.cloud.ZkCmdExecutor.retryDelay(ZkCmdExecutor.java:108)         at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:76)         at org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:291)         at org.apache.solr.core.RequestParams.getFreshRequestParams(RequestParams.java:148)         at org.apache.solr.core.SolrConfig.refreshRequestParams(SolrConfig.java:931)         at org.apache.solr.core.SolrCore$11.run(SolrCore.java:2596)         at org.apache.solr.cloud.ZkController$5.run(ZkController.java:2478)   24) Thread[id=23217, name=zkCallback-2458-thread-36-processing-n:127.0.0.1:47961_tq-EventThread, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:494)   25) Thread[id=22284, name=searcherExecutor-5006-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   26) Thread[id=20230, name=searcherExecutor-3599-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   27) Thread[id=22239, name=searcherExecutor-5027-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   28) Thread[id=22282, name=qtp1335420383-22282, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.common.cloud.ZkCmdExecutor.retryDelay(ZkCmdExecutor.java:108)         at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:76)         at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:342)         at org.apache.solr.common.cloud.ZkStateReader.fetchCollectionState(ZkStateReader.java:1035)         at org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:1020)         at org.apache.solr.common.cloud.ZkStateReader$LazyCollectionRef.get(ZkStateReader.java:555)         at org.apache.solr.common.cloud.ClusterState.getCollections(ClusterState.java:207)         at org.apache.solr.common.cloud.ZkStateReader.constructState(ZkStateReader.java:461)         at org.apache.solr.common.cloud.ZkStateReader.removeZKWatch(ZkStateReader.java:1103)         at org.apache.solr.cloud.ZkController.unregister(ZkController.java:1308)         at org.apache.solr.handler.admin.CoreAdminHandler.handleCreateAction(CoreAdminHandler.java:643)         at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestInternal(CoreAdminHandler.java:214)         at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:194)         at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:151)         at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:673)         at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:441)         at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:220)         at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:179)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:109)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.eclipse.jetty.servlets.UserAgentFilter.doFilter(UserAgentFilter.java:83)         at org.eclipse.jetty.servlets.GzipFilter.doFilter(GzipFilter.java:300)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)         at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:221)         at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)         at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)         at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)         at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)         at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)         at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)         at org.eclipse.jetty.server.Server.handle(Server.java:499)         at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)         at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)         at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)         at java.lang.Thread.run(Thread.java:745)   29) Thread[id=20237, name=searcherExecutor-3609-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   30) Thread[id=20302, name=qtp322858882-20302, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.common.cloud.ZkCmdExecutor.retryDelay(ZkCmdExecutor.java:108)         at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:76)         at org.apache.solr.common.cloud.SolrZkClient.create(SolrZkClient.java:377)         at org.apache.solr.cloud.DistributedQueue.offer(DistributedQueue.java:250)         at org.apache.solr.cloud.ZkController.unregister(ZkController.java:1315)         at org.apache.solr.handler.admin.CoreAdminHandler.handleCreateAction(CoreAdminHandler.java:643)         at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestInternal(CoreAdminHandler.java:214)         at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:194)         at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:151)         at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:673)         at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:441)         at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:220)         at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:179)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:109)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.eclipse.jetty.servlets.UserAgentFilter.doFilter(UserAgentFilter.java:83)         at org.eclipse.jetty.servlets.GzipFilter.doFilter(GzipFilter.java:300)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)         at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:221)         at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)         at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)         at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)         at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)         at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)         at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)         at org.eclipse.jetty.server.Server.handle(Server.java:499)         at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)         at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)         at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)         at java.lang.Thread.run(Thread.java:745)   31) Thread[id=22363, name=searcherExecutor-5131-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   32) Thread[id=22195, name=qtp1335420383-22195, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.common.cloud.ZkCmdExecutor.retryDelay(ZkCmdExecutor.java:108)         at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:76)         at org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:308)         at org.apache.solr.cloud.ZkController.pathExists(ZkController.java:872)         at org.apache.solr.cloud.ZkSolrResourceLoader.openResource(ZkSolrResourceLoader.java:82)         at org.apache.solr.core.SolrResourceLoader.openSchema(SolrResourceLoader.java:309)         at org.apache.solr.schema.IndexSchemaFactory.create(IndexSchemaFactory.java:47)         at org.apache.solr.schema.IndexSchemaFactory.buildIndexSchema(IndexSchemaFactory.java:69)         at org.apache.solr.core.ConfigSetService.createIndexSchema(ConfigSetService.java:107)         at org.apache.solr.core.ConfigSetService.getConfig(ConfigSetService.java:78)         at org.apache.solr.core.CoreContainer.create(CoreContainer.java:732)         at org.apache.solr.core.CoreContainer.create(CoreContainer.java:708)         at org.apache.solr.handler.admin.CoreAdminHandler.handleCreateAction(CoreAdminHandler.java:633)         at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestInternal(CoreAdminHandler.java:214)         at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:194)         at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:151)         at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:673)         at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:441)         at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:220)         at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:179)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:109)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.eclipse.jetty.servlets.UserAgentFilter.doFilter(UserAgentFilter.java:83)         at org.eclipse.jetty.servlets.GzipFilter.doFilter(GzipFilter.java:300)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)         at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:221)         at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)         at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)         at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)         at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)         at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)         at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)         at org.eclipse.jetty.server.Server.handle(Server.java:499)         at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)         at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)         at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)         at java.lang.Thread.run(Thread.java:745)   33) Thread[id=22340, name=searcherExecutor-5086-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   34) Thread[id=22347, name=searcherExecutor-5099-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   35) Thread[id=22093, name=searcherExecutor-4952-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   36) Thread[id=22193, name=qtp989002053-22193, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.common.cloud.ZkCmdExecutor.retryDelay(ZkCmdExecutor.java:108)         at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:76)         at org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:308)         at org.apache.solr.cloud.ZkController.pathExists(ZkController.java:872)         at org.apache.solr.cloud.ZkSolrResourceLoader.openResource(ZkSolrResourceLoader.java:82)         at org.apache.solr.core.SolrResourceLoader.openSchema(SolrResourceLoader.java:309)         at org.apache.solr.schema.IndexSchemaFactory.create(IndexSchemaFactory.java:47)         at org.apache.solr.schema.IndexSchemaFactory.buildIndexSchema(IndexSchemaFactory.java:69)         at org.apache.solr.core.ConfigSetService.createIndexSchema(ConfigSetService.java:107)         at org.apache.solr.core.ConfigSetService.getConfig(ConfigSetService.java:78)         at org.apache.solr.core.CoreContainer.create(CoreContainer.java:732)         at org.apache.solr.core.CoreContainer.create(CoreContainer.java:708)         at org.apache.solr.handler.admin.CoreAdminHandler.handleCreateAction(CoreAdminHandler.java:633)         at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestInternal(CoreAdminHandler.java:214)         at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:194)         at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:151)         at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:673)         at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:441)         at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:220)         at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:179)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:109)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.eclipse.jetty.servlets.UserAgentFilter.doFilter(UserAgentFilter.java:83)         at org.eclipse.jetty.servlets.GzipFilter.doFilter(GzipFilter.java:300)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)         at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:221)         at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)         at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)         at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)         at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)         at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)         at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)         at org.eclipse.jetty.server.Server.handle(Server.java:499)         at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)         at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)         at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)         at java.lang.Thread.run(Thread.java:745)   37) Thread[id=22334, name=searcherExecutor-5064-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   38) Thread[id=23179, name=zkCallback-2505-thread-44, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   39) Thread[id=23059, name=Thread-13675, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.common.cloud.ZkCmdExecutor.retryDelay(ZkCmdExecutor.java:108)         at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:76)         at org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:291)         at org.apache.solr.core.RequestParams.getFreshRequestParams(RequestParams.java:148)         at org.apache.solr.core.SolrConfig.refreshRequestParams(SolrConfig.java:931)         at org.apache.solr.core.SolrCore$11.run(SolrCore.java:2596)         at org.apache.solr.cloud.ZkController$5.run(ZkController.java:2478)   40) Thread[id=22082, name=searcherExecutor-4940-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   41) Thread[id=22604, name=searcherExecutor-5202-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   42) Thread[id=19969, name=qtp1335420383-19969-selector-ServerConnectorManager@12c3fb2f/0, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102)         at org.eclipse.jetty.io.SelectorManager$ManagedSelector.select(SelectorManager.java:600)         at org.eclipse.jetty.io.SelectorManager$ManagedSelector.run(SelectorManager.java:549)         at org.eclipse.jetty.util.thread.NonBlockingThread.run(NonBlockingThread.java:52)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)         at java.lang.Thread.run(Thread.java:745)   43) Thread[id=23192, name=zkCallback-2466-thread-36, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   44) Thread[id=20288, name=qtp1335420383-20288, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.common.cloud.ZkStateReader.getLeaderRetry(ZkStateReader.java:640)         at org.apache.solr.common.cloud.ZkStateReader.getLeaderUrl(ZkStateReader.java:609)         at org.apache.solr.cloud.ZkController.getLeader(ZkController.java:1004)         at org.apache.solr.cloud.ZkController.register(ZkController.java:939)         at org.apache.solr.cloud.ZkController.register(ZkController.java:882)         at org.apache.solr.core.ZkContainer$2.run(ZkContainer.java:183)         at org.apache.solr.core.ZkContainer.registerInZk(ZkContainer.java:212)         at org.apache.solr.core.CoreContainer.registerCore(CoreContainer.java:688)         at org.apache.solr.core.CoreContainer.create(CoreContainer.java:742)         at org.apache.solr.core.CoreContainer.create(CoreContainer.java:708)         at org.apache.solr.handler.admin.CoreAdminHandler.handleCreateAction(CoreAdminHandler.java:633)         at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestInternal(CoreAdminHandler.java:214)         at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:194)         at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:151)         at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:673)         at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:441)         at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:220)         at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:179)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:109)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.eclipse.jetty.servlets.UserAgentFilter.doFilter(UserAgentFilter.java:83)         at org.eclipse.jetty.servlets.GzipFilter.doFilter(GzipFilter.java:300)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)         at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:221)         at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)         at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)         at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)         at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)         at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)         at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)         at org.eclipse.jetty.server.Server.handle(Server.java:499)         at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)         at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)         at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)         at java.lang.Thread.run(Thread.java:745)   45) Thread[id=23216, name=zkCallback-2458-thread-36-processing-n:127.0.0.1:47961_tq-SendThread(127.0.0.1:33764), state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)         at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:940)         at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1003)   46) Thread[id=23063, name=RecoveryThread-awholynewstresscollection_collection0_5_shard9_replica2, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:519)         at org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:225)   47) Thread[id=23191, name=zkCallback-2505-thread-51, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   48) Thread[id=22980, name=Thread-13614, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.common.cloud.ZkCmdExecutor.retryDelay(ZkCmdExecutor.java:108)         at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:76)         at org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:291)         at org.apache.solr.core.RequestParams.getFreshRequestParams(RequestParams.java:148)         at org.apache.solr.core.SolrConfig.refreshRequestParams(SolrConfig.java:931)         at org.apache.solr.core.SolrCore$11.run(SolrCore.java:2596)         at org.apache.solr.cloud.ZkController$5.run(ZkController.java:2478)   49) Thread[id=22099, name=searcherExecutor-4955-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   50) Thread[id=19942, name=qtp989002053-19942-selector-ServerConnectorManager@3c6886ac/1, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102)         at org.eclipse.jetty.io.SelectorManager$ManagedSelector.select(SelectorManager.java:600)         at org.eclipse.jetty.io.SelectorManager$ManagedSelector.run(SelectorManager.java:549)         at org.eclipse.jetty.util.thread.NonBlockingThread.run(NonBlockingThread.java:52)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)         at java.lang.Thread.run(Thread.java:745)   51) Thread[id=21923, name=qtp1913862660-21923, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.common.cloud.ZkStateReader.getLeaderRetry(ZkStateReader.java:640)         at org.apache.solr.common.cloud.ZkStateReader.getLeaderUrl(ZkStateReader.java:609)         at org.apache.solr.cloud.ZkController.getLeader(ZkController.java:1004)         at org.apache.solr.cloud.ZkController.register(ZkController.java:939)         at org.apache.solr.cloud.ZkController.register(ZkController.java:882)         at org.apache.solr.core.ZkContainer$2.run(ZkContainer.java:183)         at org.apache.solr.core.ZkContainer.registerInZk(ZkContainer.java:212)         at org.apache.solr.core.CoreContainer.registerCore(CoreContainer.java:688)         at org.apache.solr.core.CoreContainer.create(CoreContainer.java:742)         at org.apache.solr.core.CoreContainer.create(CoreContainer.java:708)         at org.apache.solr.handler.admin.CoreAdminHandler.handleCreateAction(CoreAdminHandler.java:633)         at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestInternal(CoreAdminHandler.java:214)         at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:194)         at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:151)         at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:673)         at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:441)         at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:220)         at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:179)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:109)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.eclipse.jetty.servlets.UserAgentFilter.doFilter(UserAgentFilter.java:83)         at org.eclipse.jetty.servlets.GzipFilter.doFilter(GzipFilter.java:300)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)         at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:221)         at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)         at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)         at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)         at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)         at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)         at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)         at org.eclipse.jetty.server.Server.handle(Server.java:499)         at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)         at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)         at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)         at java.lang.Thread.run(Thread.java:745)   52) Thread[id=20337, name=searcherExecutor-3636-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   53) Thread[id=22084, name=searcherExecutor-4950-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   54) Thread[id=19918, name=qtp1913862660-19918, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.common.cloud.ZkStateReader.getLeaderRetry(ZkStateReader.java:640)         at org.apache.solr.common.cloud.ZkStateReader.getLeaderUrl(ZkStateReader.java:609)         at org.apache.solr.cloud.ZkController.getLeader(ZkController.java:1004)         at org.apache.solr.cloud.ZkController.register(ZkController.java:939)         at org.apache.solr.cloud.ZkController.register(ZkController.java:882)         at org.apache.solr.core.ZkContainer$2.run(ZkContainer.java:183)         at org.apache.solr.core.ZkContainer.registerInZk(ZkContainer.java:212)         at org.apache.solr.core.CoreContainer.registerCore(CoreContainer.java:688)         at org.apache.solr.core.CoreContainer.create(CoreContainer.java:742)         at org.apache.solr.core.CoreContainer.create(CoreContainer.java:708)         at org.apache.solr.handler.admin.CoreAdminHandler.handleCreateAction(CoreAdminHandler.java:633)         at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestInternal(CoreAdminHandler.java:214)         at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:194)         at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:151)         at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:673)         at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:441)         at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:220)         at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:179)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:109)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.eclipse.jetty.servlets.UserAgentFilter.doFilter(UserAgentFilter.java:83)         at org.eclipse.jetty.servlets.GzipFilter.doFilter(GzipFilter.java:300)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)         at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:221)         at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)         at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)         at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)         at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)         at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)         at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)         at org.eclipse.jetty.server.Server.handle(Server.java:499)         at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)         at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)         at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)         at java.lang.Thread.run(Thread.java:745)   55) Thread[id=21974, name=qtp322858882-21974, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.common.cloud.ZkStateReader.getLeaderRetry(ZkStateReader.java:640)         at org.apache.solr.common.cloud.ZkStateReader.getLeaderUrl(ZkStateReader.java:609)         at org.apache.solr.cloud.ZkController.getLeader(ZkController.java:1004)         at org.apache.solr.cloud.ZkController.register(ZkController.java:939)         at org.apache.solr.cloud.ZkController.register(ZkController.java:882)         at org.apache.solr.core.ZkContainer$2.run(ZkContainer.java:183)         at org.apache.solr.core.ZkContainer.registerInZk(ZkContainer.java:212)         at org.apache.solr.core.CoreContainer.registerCore(CoreContainer.java:688)         at org.apache.solr.core.CoreContainer.create(CoreContainer.java:742)         at org.apache.solr.core.CoreContainer.create(CoreContainer.java:708)         at org.apache.solr.handler.admin.CoreAdminHandler.handleCreateAction(CoreAdminHandler.java:633)         at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestInternal(CoreAdminHandler.java:214)         at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:194)         at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:151)         at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:673)         at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:441)         at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:220)         at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:179)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:109)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.eclipse.jetty.servlets.UserAgentFilter.doFilter(UserAgentFilter.java:83)         at org.eclipse.jetty.servlets.GzipFilter.doFilter(GzipFilter.java:300)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)         at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:221)         at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)         at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)         at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)         at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)         at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)         at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)         at org.eclipse.jetty.server.Server.handle(Server.java:499)         at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)         at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)         at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)         at java.lang.Thread.run(Thread.java:745)   56) Thread[id=19986, name=TEST-CollectionsAPIDistributedZkTest.test-seed#[F01B49D9BE6011D0]-EventThread, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:494)   57) Thread[id=22060, name=searcherExecutor-4911-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   58) Thread[id=23198, name=zkCallback-2505-thread-55, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   59) Thread[id=22067, name=searcherExecutor-4923-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   60) Thread[id=22196, name=qtp322858882-22196, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.common.cloud.ZkCmdExecutor.retryDelay(ZkCmdExecutor.java:108)         at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:76)         at org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:308)         at org.apache.solr.cloud.ZkController.pathExists(ZkController.java:872)         at org.apache.solr.cloud.ZkSolrResourceLoader.openResource(ZkSolrResourceLoader.java:82)         at org.apache.solr.core.SolrResourceLoader.openSchema(SolrResourceLoader.java:309)         at org.apache.solr.schema.IndexSchemaFactory.create(IndexSchemaFactory.java:47)         at org.apache.solr.schema.IndexSchemaFactory.buildIndexSchema(IndexSchemaFactory.java:69)         at org.apache.solr.core.ConfigSetService.createIndexSchema(ConfigSetService.java:107)         at org.apache.solr.core.ConfigSetService.getConfig(ConfigSetService.java:78)         at org.apache.solr.core.CoreContainer.create(CoreContainer.java:73

[...truncated too long message...]

PoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >   173) Thread[id=22057, name=searcherExecutor-4904-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
   [junit4]    >         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >   174) Thread[id=23202, name=zkCallback-2505-thread-59, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)
   [junit4]    >         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >   175) Thread[id=22201, name=searcherExecutor-5001-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
   [junit4]    >         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >   176) Thread[id=20353, name=searcherExecutor-3652-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
   [junit4]    >         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >   177) Thread[id=22380, name=searcherExecutor-5110-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
   [junit4]    >         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >   178) Thread[id=22358, name=searcherExecutor-5097-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
   [junit4]    >         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >   179) Thread[id=22359, name=searcherExecutor-5108-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
   [junit4]    >         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >   180) Thread[id=23195, name=zkCallback-2505-thread-52, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)
   [junit4]    >         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([F01B49D9BE6011D0]:0)
   [junit4] Completed [286/547] on J1 in 1229.44s, 1 test, 3 errors <<< FAILURES!

[...truncated 777 lines...]
   [junit4] JVM J1: stdout was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J1-20151116_223518_659.sysout
   [junit4] >>> JVM J1: stdout (verbatim) ----
   [junit4] java.lang.OutOfMemoryError: GC overhead limit exceeded
   [junit4] Dumping heap to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/heapdumps/java_pid32393.hprof ...
   [junit4] Heap dump file created [611051036 bytes in 9.747 secs]
   [junit4] <<< JVM J1: EOF ----

[...truncated 18 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 547 suites (7 ignored), 2163 tests, 2 suite-level errors, 1 error, 820 ignored (14 assumptions) [seed: F01B49D9BE6011D0]

Total time: 167 minutes 33 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1018 - Still Failing

[Apache Jenkins Server <je...@builds.apache.org>]

Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1018/

1 tests failed.
FAILED:  org.apache.solr.cloud.hdfs.HdfsBasicDistributedZk2Test.test

Error Message:


Stack Trace:
java.lang.NullPointerException
	at __randomizedtesting.SeedInfo.seed([D9C7AF4E9AE9B76C:519390943415DA94]:0)
	at org.apache.solr.client.solrj.embedded.JettySolrRunner.getCoreContainer(JettySolrRunner.java:373)
	at org.apache.solr.cloud.ChaosMonkey.expireSession(ChaosMonkey.java:122)
	at org.apache.solr.cloud.BasicDistributedZk2Test.test(BasicDistributedZk2Test.java:131)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1660)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:866)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:902)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:916)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:963)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:938)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:875)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:777)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:811)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:822)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 11491 lines...]
   [junit4] Suite: org.apache.solr.cloud.hdfs.HdfsBasicDistributedZk2Test
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_D9C7AF4E9AE9B76C-001/init-core-data-001
   [junit4]   2> 3007393 INFO  (SUITE-HdfsBasicDistributedZk2Test-seed#[D9C7AF4E9AE9B76C]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /_jyi/q
   [junit4]   1> Formatting using clusterid: testClusterID
   [junit4]   2> 3007427 WARN  (SUITE-HdfsBasicDistributedZk2Test-seed#[D9C7AF4E9AE9B76C]-worker) [    ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
   [junit4]   2> 3007433 WARN  (SUITE-HdfsBasicDistributedZk2Test-seed#[D9C7AF4E9AE9B76C]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 3007437 INFO  (SUITE-HdfsBasicDistributedZk2Test-seed#[D9C7AF4E9AE9B76C]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 3007450 INFO  (SUITE-HdfsBasicDistributedZk2Test-seed#[D9C7AF4E9AE9B76C]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/hdfs to ./temp/Jetty_localhost_43826_hdfs____.hhgibd/webapp
   [junit4]   2> 3007579 INFO  (SUITE-HdfsBasicDistributedZk2Test-seed#[D9C7AF4E9AE9B76C]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 3007772 INFO  (SUITE-HdfsBasicDistributedZk2Test-seed#[D9C7AF4E9AE9B76C]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:43826
   [junit4]   2> 3007830 WARN  (SUITE-HdfsBasicDistributedZk2Test-seed#[D9C7AF4E9AE9B76C]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 3007832 INFO  (SUITE-HdfsBasicDistributedZk2Test-seed#[D9C7AF4E9AE9B76C]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 3007845 INFO  (SUITE-HdfsBasicDistributedZk2Test-seed#[D9C7AF4E9AE9B76C]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_37143_datanode____.kds499/webapp
   [junit4]   2> 3007965 INFO  (SUITE-HdfsBasicDistributedZk2Test-seed#[D9C7AF4E9AE9B76C]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 3008173 INFO  (SUITE-HdfsBasicDistributedZk2Test-seed#[D9C7AF4E9AE9B76C]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:37143
   [junit4]   2> 3008205 WARN  (SUITE-HdfsBasicDistributedZk2Test-seed#[D9C7AF4E9AE9B76C]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 3008206 INFO  (SUITE-HdfsBasicDistributedZk2Test-seed#[D9C7AF4E9AE9B76C]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 3008220 INFO  (SUITE-HdfsBasicDistributedZk2Test-seed#[D9C7AF4E9AE9B76C]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_40935_datanode____.owpge2/webapp
   [junit4]   2> 3008268 INFO  (IPC Server handler 3 on 51807) [    ] BlockStateChange BLOCK* processReport: from storage DS-0c4ef819-0165-4dbe-bd74-0f73b8bc3238 node DatanodeRegistration(127.0.0.1, datanodeUuid=dd4f7226-de59-4515-9dc7-6c438025e8f7, infoPort=37143, ipcPort=38847, storageInfo=lv=-56;cid=testClusterID;nsid=758211504;c=0), blocks: 0, hasStaleStorages: true, processing time: 1 msecs
   [junit4]   2> 3008268 INFO  (IPC Server handler 3 on 51807) [    ] BlockStateChange BLOCK* processReport: from storage DS-426ffcbe-773f-45cd-b671-67cb33db5b82 node DatanodeRegistration(127.0.0.1, datanodeUuid=dd4f7226-de59-4515-9dc7-6c438025e8f7, infoPort=37143, ipcPort=38847, storageInfo=lv=-56;cid=testClusterID;nsid=758211504;c=0), blocks: 0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 3008367 INFO  (SUITE-HdfsBasicDistributedZk2Test-seed#[D9C7AF4E9AE9B76C]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 3008599 INFO  (SUITE-HdfsBasicDistributedZk2Test-seed#[D9C7AF4E9AE9B76C]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:40935
   [junit4]   2> 3008680 INFO  (IPC Server handler 8 on 51807) [    ] BlockStateChange BLOCK* processReport: from storage DS-caddcb5b-de83-454d-8270-60cf1eb69806 node DatanodeRegistration(127.0.0.1, datanodeUuid=f75118f0-e840-4aff-bed1-ced4a9456950, infoPort=40935, ipcPort=49651, storageInfo=lv=-56;cid=testClusterID;nsid=758211504;c=0), blocks: 0, hasStaleStorages: true, processing time: 0 msecs
   [junit4]   2> 3008681 INFO  (IPC Server handler 8 on 51807) [    ] BlockStateChange BLOCK* processReport: from storage DS-9fbb1db4-b323-4fca-8206-f00521017c6b node DatanodeRegistration(127.0.0.1, datanodeUuid=f75118f0-e840-4aff-bed1-ced4a9456950, infoPort=40935, ipcPort=49651, storageInfo=lv=-56;cid=testClusterID;nsid=758211504;c=0), blocks: 0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 3008776 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 3008776 INFO  (Thread-15984) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 3008776 INFO  (Thread-15984) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 3008876 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.ZkTestServer start zk server on port:43636
   [junit4]   2> 3008876 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3008877 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3008885 INFO  (zkCallback-3019-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@30329923 name:ZooKeeperConnection Watcher:127.0.0.1:43636 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3008885 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3008886 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3008886 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 3008890 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3008890 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3008891 INFO  (zkCallback-3020-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1fb5980b name:ZooKeeperConnection Watcher:127.0.0.1:43636/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3008891 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3008892 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3008892 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 3008894 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 3008896 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 3008897 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 3008899 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 3008899 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 3008902 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 3008902 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 3008904 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 3008904 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 3008906 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 3008906 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 3008908 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 3008908 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 3008910 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 3008910 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 3008912 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 3008912 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 3008913 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 3008913 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 3008915 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 3008915 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 3008917 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 3008917 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 3008919 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 3008919 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 3009212 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_D9C7AF4E9AE9B76C-001/control-001/cores/collection1
   [junit4]   2> 3009213 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 3009221 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@557917d5{/_jyi/q,null,AVAILABLE}
   [junit4]   2> 3009221 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.e.j.s.ServerConnector Started ServerConnector@2db1e24f{HTTP/1.1}{127.0.0.1:59584}
   [junit4]   2> 3009221 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.e.j.s.Server Started @3011834ms
   [junit4]   2> 3009221 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=hdfs://localhost:51807/hdfs__localhost_51807__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-5.x_solr_build_solr-core_test_J1_temp_solr.cloud.hdfs.HdfsBasicDistributedZk2Test_D9C7AF4E9AE9B76C-001_tempDir-002_control_data, hostContext=/_jyi/q, hostPort=59584, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_D9C7AF4E9AE9B76C-001/control-001/cores}
   [junit4]   2> 3009222 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@7b3cb2c6
   [junit4]   2> 3009222 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_D9C7AF4E9AE9B76C-001/control-001/'
   [junit4]   2> 3009222 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 3009222 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 3009239 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3009241 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3009242 INFO  (zkCallback-3021-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@795d7d06 name:ZooKeeperConnection Watcher:127.0.0.1:43636/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3009242 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3009243 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3009244 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 3009244 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_D9C7AF4E9AE9B76C-001/control-001/solr.xml
   [junit4]   2> 3009268 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_D9C7AF4E9AE9B76C-001/control-001/cores
   [junit4]   2> 3009268 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.CoreContainer New CoreContainer 256502587
   [junit4]   2> 3009268 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_D9C7AF4E9AE9B76C-001/control-001/]
   [junit4]   2> 3009268 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_D9C7AF4E9AE9B76C-001/control-001/lib
   [junit4]   2> 3009269 WARN  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_D9C7AF4E9AE9B76C-001/control-001/lib).
   [junit4]   2> 3009296 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 3009297 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 3009298 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 3009298 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 3009298 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:43636/solr
   [junit4]   2> 3009298 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 3009298 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3009299 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3009300 INFO  (zkCallback-3023-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@42c9f8d0 name:ZooKeeperConnection Watcher:127.0.0.1:43636 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3009300 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3009301 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3009302 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [n:127.0.0.1:59584__jyi%2Fq    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3009303 INFO  (zkCallback-3024-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@75028438 name:ZooKeeperConnection Watcher:127.0.0.1:43636/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3009303 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [n:127.0.0.1:59584__jyi%2Fq    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3009304 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [n:127.0.0.1:59584__jyi%2Fq    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 3009306 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [n:127.0.0.1:59584__jyi%2Fq    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 3009309 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [n:127.0.0.1:59584__jyi%2Fq    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 3009311 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [n:127.0.0.1:59584__jyi%2Fq    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 3009313 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [n:127.0.0.1:59584__jyi%2Fq    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 3009315 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [n:127.0.0.1:59584__jyi%2Fq    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 3009317 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [n:127.0.0.1:59584__jyi%2Fq    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 3009318 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [n:127.0.0.1:59584__jyi%2Fq    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 3009319 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [n:127.0.0.1:59584__jyi%2Fq    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 3009320 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [n:127.0.0.1:59584__jyi%2Fq    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:59584__jyi%2Fq
   [junit4]   2> 3009320 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [n:127.0.0.1:59584__jyi%2Fq    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:59584__jyi%2Fq
   [junit4]   2> 3009322 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [n:127.0.0.1:59584__jyi%2Fq    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 3009323 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [n:127.0.0.1:59584__jyi%2Fq    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 3009324 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [n:127.0.0.1:59584__jyi%2Fq    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 3009325 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [n:127.0.0.1:59584__jyi%2Fq    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94872104019099652-127.0.0.1:59584__jyi%2Fq-n_0000000000
   [junit4]   2> 3009325 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [n:127.0.0.1:59584__jyi%2Fq    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:59584__jyi%2Fq
   [junit4]   2> 3009325 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [n:127.0.0.1:59584__jyi%2Fq    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 3009326 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [n:127.0.0.1:59584__jyi%2Fq    ] o.a.s.c.Overseer Overseer (id=94872104019099652-127.0.0.1:59584__jyi%2Fq-n_0000000000) starting
   [junit4]   2> 3009328 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [n:127.0.0.1:59584__jyi%2Fq    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 3009334 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [n:127.0.0.1:59584__jyi%2Fq    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 3009334 INFO  (OverseerCollectionConfigSetProcessor-94872104019099652-127.0.0.1:59584__jyi%2Fq-n_0000000000) [n:127.0.0.1:59584__jyi%2Fq    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 3009335 INFO  (OverseerStateUpdate-94872104019099652-127.0.0.1:59584__jyi%2Fq-n_0000000000) [n:127.0.0.1:59584__jyi%2Fq    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 3009335 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [n:127.0.0.1:59584__jyi%2Fq    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 3009352 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [n:127.0.0.1:59584__jyi%2Fq    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 3009352 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [n:127.0.0.1:59584__jyi%2Fq    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 3009353 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [n:127.0.0.1:59584__jyi%2Fq    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_D9C7AF4E9AE9B76C-001/control-001/cores
   [junit4]   2> 3009354 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [n:127.0.0.1:59584__jyi%2Fq    ] o.a.s.c.SolrCore Created CoreDescriptor: {name=collection1, shard=, dataDir=data/, config=solrconfig.xml, configSetProperties=configsetprops.json, schema=schema.xml, transient=false, collection=control_collection, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_D9C7AF4E9AE9B76C-001/control-001/cores/collection1, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_D9C7AF4E9AE9B76C-001/control-001/cores/collection1/, coreNodeName=, loadOnStartup=true}
   [junit4]   2> 3009354 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [n:127.0.0.1:59584__jyi%2Fq    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_D9C7AF4E9AE9B76C-001/control-001/cores/collection1/
   [junit4]   2> 3009354 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [n:127.0.0.1:59584__jyi%2Fq    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 3009356 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 3009356 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 3009356 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [n:127.0.0.1:59584__jyi%2Fq    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 3009356 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [n:127.0.0.1:59584__jyi%2Fq    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 3009358 INFO  (OverseerStateUpdate-94872104019099652-127.0.0.1:59584__jyi%2Fq-n_0000000000) [n:127.0.0.1:59584__jyi%2Fq    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "node_name":"127.0.0.1:59584__jyi%2Fq",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "base_url":"http://127.0.0.1:59584/_jyi/q",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "state":"down",
   [junit4]   2>   "roles":null} current state version: 0
   [junit4]   2> 3009358 INFO  (OverseerStateUpdate-94872104019099652-127.0.0.1:59584__jyi%2Fq-n_0000000000) [n:127.0.0.1:59584__jyi%2Fq    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "node_name":"127.0.0.1:59584__jyi%2Fq",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "base_url":"http://127.0.0.1:59584/_jyi/q",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "state":"down",
   [junit4]   2>   "roles":null}
   [junit4]   2> 3009358 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 3009358 INFO  (OverseerStateUpdate-94872104019099652-127.0.0.1:59584__jyi%2Fq-n_0000000000) [n:127.0.0.1:59584__jyi%2Fq    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 3009359 INFO  (OverseerStateUpdate-94872104019099652-127.0.0.1:59584__jyi%2Fq-n_0000000000) [n:127.0.0.1:59584__jyi%2Fq    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 3009361 INFO  (zkCallback-3024-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3010359 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 3010359 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 3010360 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 3010360 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 3010360 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 3010361 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_D9C7AF4E9AE9B76C-001/control-001/cores/collection1/'
   [junit4]   2> 3010361 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 3010361 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 3010376 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 3010382 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 3010395 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 3010422 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 3010423 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 3010429 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 3010666 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 3010673 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 3010675 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 3010684 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 3010686 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 3010689 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 3010689 WARN  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 3010690 WARN  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 3010690 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 3010690 WARN  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 3010690 WARN  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 3010691 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 3010691 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 3010691 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.HdfsDirectoryFactory
   [junit4]   2> 3010692 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:51807/solr_hdfs_home
   [junit4]   2> 3010692 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
   [junit4]   2> 3010692 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_D9C7AF4E9AE9B76C-001/control-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 3010692 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@4f085760
   [junit4]   2> 3010692 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:51807/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 3010707 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:51807/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 3010708 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=hdfs://localhost:51807/solr_hdfs_home/control_collection/core_node1/data/index/
   [junit4]   2> 3010718 WARN  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory 'hdfs:/localhost:51807/solr_hdfs_home/control_collection/core_node1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 3010718 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:51807/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 3010726 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
   [junit4]   2> 3010726 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
   [junit4]   2> 3010726 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
   [junit4]   2> 3010733 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
   [junit4]   2> 3010733 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:51807/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 3010734 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=41, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
   [junit4]   2> 3010750 INFO  (IPC Server handler 5 on 51807) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:38025 is added to blk_1073741825_1001{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-caddcb5b-de83-454d-8270-60cf1eb69806:NORMAL:127.0.0.1:44092|RBW], ReplicaUnderConstruction[[DISK]DS-0c4ef819-0165-4dbe-bd74-0f73b8bc3238:NORMAL:127.0.0.1:38025|RBW]]} size 0
   [junit4]   2> 3010751 INFO  (IPC Server handler 4 on 51807) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:44092 is added to blk_1073741825_1001{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-0c4ef819-0165-4dbe-bd74-0f73b8bc3238:NORMAL:127.0.0.1:38025|RBW], ReplicaUnderConstruction[[DISK]DS-9fbb1db4-b323-4fca-8206-f00521017c6b:NORMAL:127.0.0.1:44092|FINALIZED]]} size 0
   [junit4]   2> 3010754 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(BlockDirectory(HdfsDirectory@b82e05b9 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@31d29ab8); maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 3010754 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 3010755 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in hdfs://localhost:51807/solr_hdfs_home/control_collection/core_node1/data/
   [junit4]   2> 3010759 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 3010759 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 3010760 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 3010760 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 3010760 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 3010760 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 3010760 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 3010760 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 3010760 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 3010761 WARN  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
   [junit4]   2> 3010763 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 3010764 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 3010766 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 3010767 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 3010771 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/mbeans,/admin/file,/update,/update/json,/admin/segments,/admin/system,/admin/ping,/admin/luke,/replication,/admin/threads,/dump,/update/csv,/get,/admin/logging,/admin/properties,/schema,/config,/admin/plugins,standard,/update/json/docs
   [junit4]   2> 3010773 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 3010773 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
   [junit4]   2> 3010773 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing HdfsUpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 tlogDfsReplication=2
   [junit4]   2> 3010783 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 3010783 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 3010785 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@34a84739
   [junit4]   2> 3010791 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(BlockDirectory(HdfsDirectory@b82e05b9 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@31d29ab8); maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 3010791 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 3010791 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@55c5f9d9[collection1] main
   [junit4]   2> 3010791 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 3010792 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 3010792 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 3010792 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 3010793 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 3010793 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 3010793 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 3010793 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 3010793 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 3010793 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 3010794 INFO  (searcherExecutor-6697-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq c:control_collection x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@55c5f9d9[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 3010795 WARN  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.h.HdfsLocalityReporter Could not retrieve locality information for hdfs://localhost:51816/solr3 due to exception: java.net.ConnectException: Call From lucene1-us-west/10.41.0.5 to localhost:51816 failed on connection exception: java.net.ConnectException: Connection refused; For more details see:  http://wiki.apache.org/hadoop/ConnectionRefused
   [junit4]   2> 3010795 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 3010795 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 256 version buckets from index
   [junit4]   2> 3010795 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 3010795 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1517953666420572160
   [junit4]   2> 3010796 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 0.0ms to seed version buckets with highest version 1517953666420572160
   [junit4]   2> 3010796 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 3010796 INFO  (coreLoadExecutor-6696-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 3010797 INFO  (coreZkRegister-6690-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq c:control_collection x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:59584/_jyi/q collection:control_collection shard:shard1
   [junit4]   2> 3010797 INFO  (coreZkRegister-6690-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq c:control_collection x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 3010800 INFO  (coreZkRegister-6690-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq c:control_collection x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94872104019099652-core_node1-n_0000000000
   [junit4]   2> 3010801 INFO  (coreZkRegister-6690-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq c:control_collection x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 3010802 INFO  (coreZkRegister-6690-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq c:control_collection x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 3010802 INFO  (coreZkRegister-6690-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq c:control_collection x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 3010802 INFO  (coreZkRegister-6690-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq c:control_collection x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:59584/_jyi/q/collection1/
   [junit4]   2> 3010802 INFO  (coreZkRegister-6690-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq c:control_collection x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 3010802 INFO  (coreZkRegister-6690-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq c:control_collection x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:59584/_jyi/q/collection1/ has no replicas
   [junit4]   2> 3010802 INFO  (OverseerStateUpdate-94872104019099652-127.0.0.1:59584__jyi%2Fq-n_0000000000) [n:127.0.0.1:59584__jyi%2Fq    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 3010803 INFO  (coreZkRegister-6690-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq c:control_collection x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 3010805 INFO  (coreZkRegister-6690-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq c:control_collection x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 3010807 INFO  (coreZkRegister-6690-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq c:control_collection x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:59584/_jyi/q/collection1/ shard1
   [junit4]   2> 3010808 INFO  (OverseerStateUpdate-94872104019099652-127.0.0.1:59584__jyi%2Fq-n_0000000000) [n:127.0.0.1:59584__jyi%2Fq    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:59584/_jyi/q",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 1
   [junit4]   2> 3010858 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3010859 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3010860 INFO  (zkCallback-3026-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@15656154 name:ZooKeeperConnection Watcher:127.0.0.1:43636/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3010860 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3010860 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3010860 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 3010862 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 3010862 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with stateFormat=2
   [junit4]   2> 3010863 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3010863 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3010864 INFO  (zkCallback-3027-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@25439679 name:ZooKeeperConnection Watcher:127.0.0.1:43636/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3010864 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3010864 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3010867 INFO  (OverseerStateUpdate-94872104019099652-127.0.0.1:59584__jyi%2Fq-n_0000000000) [n:127.0.0.1:59584__jyi%2Fq    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"create",
   [junit4]   2>   "name":"collection1",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "stateFormat":"2"} current state version: 1
   [junit4]   2> 3010868 INFO  (OverseerStateUpdate-94872104019099652-127.0.0.1:59584__jyi%2Fq-n_0000000000) [n:127.0.0.1:59584__jyi%2Fq    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 3010868 INFO  (zkCallback-3026-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3010868 INFO  (zkCallback-3024-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3010908 INFO  (coreZkRegister-6690-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq c:control_collection x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:59584/_jyi/q/collection1/ and leader is http://127.0.0.1:59584/_jyi/q/collection1/
   [junit4]   2> 3010908 INFO  (coreZkRegister-6690-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq c:control_collection x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:59584/_jyi/q
   [junit4]   2> 3010908 INFO  (coreZkRegister-6690-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq c:control_collection x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 3010908 INFO  (coreZkRegister-6690-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq c:control_collection x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 3010908 INFO  (coreZkRegister-6690-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq c:control_collection x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:59584__jyi%2Fq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 3010910 INFO  (OverseerStateUpdate-94872104019099652-127.0.0.1:59584__jyi%2Fq-n_0000000000) [n:127.0.0.1:59584__jyi%2Fq    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "node_name":"127.0.0.1:59584__jyi%2Fq",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "base_url":"http://127.0.0.1:59584/_jyi/q",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "roles":null} current state version: 2
   [junit4]   2> 3010910 INFO  (OverseerStateUpdate-94872104019099652-127.0.0.1:59584__jyi%2Fq-n_0000000000) [n:127.0.0.1:59584__jyi%2Fq    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "node_name":"127.0.0.1:59584__jyi%2Fq",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "base_url":"http://127.0.0.1:59584/_jyi/q",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "roles":null}
   [junit4]   2> 3010911 INFO  (OverseerStateUpdate-94872104019099652-127.0.0.1:59584__jyi%2Fq-n_0000000000) [n:127.0.0.1:59584__jyi%2Fq    ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/collection1/state.json
   [junit4]   2> 3011015 INFO  (zkCallback-3026-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3011015 INFO  (zkCallback-3024-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3011159 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_D9C7AF4E9AE9B76C-001/shard-1-001/cores/collection1
   [junit4]   2> 3011160 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_D9C7AF4E9AE9B76C-001/shard-1-001
   [junit4]   2> 3011160 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 3011162 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@3997cca9{/_jyi/q,null,AVAILABLE}
   [junit4]   2> 3011163 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.e.j.s.ServerConnector Started ServerConnector@491f52fd{HTTP/1.1}{127.0.0.1:38016}
   [junit4]   2> 3011163 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.e.j.s.Server Started @3013776ms
   [junit4]   2> 3011163 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=hdfs://localhost:51807/hdfs__localhost_51807__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-5.x_solr_build_solr-core_test_J1_temp_solr.cloud.hdfs.HdfsBasicDistributedZk2Test_D9C7AF4E9AE9B76C-001_tempDir-002_jetty1, hostPort=38016, hostContext=/_jyi/q, solrconfig=solrconfig.xml, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_D9C7AF4E9AE9B76C-001/shard-1-001/cores}
   [junit4]   2> 3011163 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@7b3cb2c6
   [junit4]   2> 3011163 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_D9C7AF4E9AE9B76C-001/shard-1-001/'
   [junit4]   2> 3011163 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 3011163 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 3011180 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3011180 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3011181 INFO  (zkCallback-3028-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@56817f5a name:ZooKeeperConnection Watcher:127.0.0.1:43636/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3011182 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3011182 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3011183 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 3011183 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_D9C7AF4E9AE9B76C-001/shard-1-001/solr.xml
   [junit4]   2> 3011197 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_D9C7AF4E9AE9B76C-001/shard-1-001/cores
   [junit4]   2> 3011197 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.CoreContainer New CoreContainer 593435846
   [junit4]   2> 3011197 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_D9C7AF4E9AE9B76C-001/shard-1-001/]
   [junit4]   2> 3011197 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_D9C7AF4E9AE9B76C-001/shard-1-001/lib
   [junit4]   2> 3011197 WARN  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_D9C7AF4E9AE9B76C-001/shard-1-001/lib).
   [junit4]   2> 3011205 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 3011206 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 3011207 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 3011207 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 3011207 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:43636/solr
   [junit4]   2> 3011207 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 3011207 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3011208 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3011209 INFO  (zkCallback-3030-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@77658cc4 name:ZooKeeperConnection Watcher:127.0.0.1:43636 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3011209 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3011209 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3011211 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [n:127.0.0.1:38016__jyi%2Fq    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3011212 INFO  (zkCallback-3031-thread-1-processing-n:127.0.0.1:38016__jyi%2Fq) [n:127.0.0.1:38016__jyi%2Fq    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1dd9acd8 name:ZooKeeperConnection Watcher:127.0.0.1:43636/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3011212 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [n:127.0.0.1:38016__jyi%2Fq    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3011217 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [n:127.0.0.1:38016__jyi%2Fq    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 3012223 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [n:127.0.0.1:38016__jyi%2Fq    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:38016__jyi%2Fq
   [junit4]   2> 3012223 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [n:127.0.0.1:38016__jyi%2Fq    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:38016__jyi%2Fq
   [junit4]   2> 3012224 INFO  (zkCallback-3024-thread-1-processing-n:127.0.0.1:59584__jyi%2Fq) [n:127.0.0.1:59584__jyi%2Fq    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3012224 INFO  (zkCallback-3031-thread-1-processing-n:127.0.0.1:38016__jyi%2Fq) [n:127.0.0.1:38016__jyi%2Fq    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3012224 INFO  (zkCallback-3026-thread-1) [    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3012226 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [n:127.0.0.1:38016__jyi%2Fq    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 3012227 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [n:127.0.0.1:38016__jyi%2Fq    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94872104019099657-127.0.0.1:38016__jyi%2Fq-n_0000000001
   [junit4]   2> 3012228 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [n:127.0.0.1:38016__jyi%2Fq    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/94872104019099652-127.0.0.1:59584__jyi%2Fq-n_0000000000 to know if I could be the leader
   [junit4]   2> 3012238 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [n:127.0.0.1:38016__jyi%2Fq    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 3012238 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [n:127.0.0.1:38016__jyi%2Fq    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 3012238 INFO  (TEST-HdfsBasicDistributedZk2Test.test-seed#[D9C7AF4E9AE9B76C]) [n:127.0.0.1:38016__jyi%2Fq    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_D9C7AF4E9AE9B76C-001/shard-1-001/cores
   [juni

[...truncated too long message...]

sorImpl.java:57)
   [junit4]   2> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2> 	at java.lang.reflect.Method.invoke(Method.java:606)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1660)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:829)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> Caused by: java.lang.NullPointerException
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.blockmanagement.BlocksMap.size(BlocksMap.java:198)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.getTotalBlocks(BlockManager.java:3291)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlocksTotal(FSNamesystem.java:6223)
   [junit4]   2> 	... 54 more
   [junit4]   2> 3109051 INFO  (SUITE-HdfsBasicDistributedZk2Test-seed#[D9C7AF4E9AE9B76C]-worker) [n:127.0.0.1:57352__jyi%2Fq c:collection1 s:shard1 r:core_node4 x:collection1] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_D9C7AF4E9AE9B76C-001
   [junit4]   2> Nov 16, 2015 12:25:23 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 3 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=CheapBastard, sim=RandomSimilarityProvider(queryNorm=false,coord=no): {}, locale=sl, timezone=Africa/Libreville
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_80 (64-bit)/cpus=4,threads=2,free=168258384,total=527958016
   [junit4]   2> NOTE: All tests run in this JVM: [RankQueryTest, IgnoreCommitOptimizeUpdateProcessorFactoryTest, TestFieldResource, TestConfigSetImmutable, SpellCheckCollatorTest, LeaderInitiatedRecoveryOnCommitTest, TestEmbeddedSolrServerConstructors, BufferStoreTest, TestExclusionRuleCollectionAccess, TestStressVersions, CursorMarkTest, TestCloudManagedSchema, BJQParserTest, TestJsonFacets, TestDistributedGrouping, TestFileDictionaryLookup, SoftAutoCommitTest, LeaderFailoverAfterPartitionTest, TestDefaultStatsCache, AnalyticsMergeStrategyTest, TermVectorComponentTest, DocValuesTest, MoreLikeThisHandlerTest, MultiTermTest, TestMiniSolrCloudClusterSSL, TestSortingResponseWriter, DeleteLastCustomShardedReplicaTest, ShardRoutingTest, RequestLoggingTest, TestOverriddenPrefixQueryForCustomFieldType, TestSolrDynamicMBean, TestSolrCoreProperties, NotRequiredUniqueKeyTest, SpatialRPTFieldTypeTest, MBeansHandlerTest, SimplePostToolTest, TestJettySolrRunner, TestCodecSupport, TestAddFieldRealTimeGet, AnalyticsQueryTest, SolrIndexSplitterTest, OutOfBoxZkACLAndCredentialsProvidersTest, TestUpdate, SortByFunctionTest, BitVectorTest, TestFastLRUCache, TestBM25SimilarityFactory, SuggesterFSTTest, FileUtilsTest, SuggestComponentTest, TestCustomDocTransformer, TestDFRSimilarityFactory, TestSolrConfigHandlerCloud, TestSolrQueryParserDefaultOperatorResource, TestSurroundQueryParser, ShardRoutingCustomTest, DistributedTermsComponentTest, TestRestManager, CloneFieldUpdateProcessorFactoryTest, TestReRankQParserPlugin, HdfsSyncSliceTest, SolrPluginUtilsTest, ChaosMonkeyNothingIsSafeTest, TestHashQParserPlugin, FacetPivotSmallTest, HdfsDirectoryFactoryTest, BasicFunctionalityTest, HdfsChaosMonkeyNothingIsSafeTest, AddBlockUpdateTest, RestartWhileUpdatingTest, TestLeaderInitiatedRecoveryThread, SmileWriterTest, SecurityConfHandlerTest, FastVectorHighlighterTest, TestCoreContainer, DistanceFunctionTest, ClusterStateUpdateTest, TestStandardQParsers, TestHashPartitioner, SchemaVersionSpecificBehaviorTest, PreAnalyzedUpdateProcessorTest, HighlighterTest, VMParamsZkACLAndCredentialsProvidersTest, DirectSolrSpellCheckerTest, UniqFieldsUpdateProcessorFactoryTest, TestSystemIdResolver, TestStressUserVersions, BasicDistributedZkTest, UnloadDistributedZkTest, SyncSliceTest, OpenCloseCoreStressTest, TestFaceting, DistributedSpellCheckComponentTest, TermVectorComponentDistributedTest, ZkControllerTest, TestRealTimeGet, TestRangeQuery, SimpleFacetsTest, TestSolr4Spatial, StatsComponentTest, SpellCheckComponentTest, TestGroupingSearch, SolrCmdDistributorTest, PeerSyncTest, ConvertedLegacyTest, TestSort, TestFiltering, DirectUpdateHandlerTest, TestIndexSearcher, CurrencyFieldOpenExchangeTest, SignatureUpdateProcessorFactoryTest, TestStressLucene, TestTrie, SpatialFilterTest, TestCSVLoader, WordBreakSolrSpellCheckerTest, FieldMutatingUpdateProcessorTest, TestWordDelimiterFilterFactory, QueryEqualityTest, StatelessScriptUpdateProcessorFactoryTest, XsltUpdateRequestHandlerTest, IndexBasedSpellCheckerTest, TestReversedWildcardFilterFactory, TestQueryTypes, TermsComponentTest, TestSolrQueryParser, LoggingHandlerTest, ReturnFieldsTest, JSONWriterTest, JsonLoaderTest, PingRequestHandlerTest, HighlighterConfigTest, SOLR749Test, TestQuerySenderListener, TestQuerySenderNoQuery, CopyFieldTest, ResponseLogComponentTest, SolrIndexConfigTest, TestSolrDeletionPolicy2, SampleTest, TestBinaryField, NumericFieldsTest, MinimalSchemaTest, TestElisionMultitermQuery, ExternalFileFieldSortTest, TestLuceneMatchVersion, TestPhraseSuggestions, SpellPossibilityIteratorTest, SynonymTokenizerTest, TestXIncludeConfig, EchoParamsTest, TestSweetSpotSimilarityFactory, TestLMDirichletSimilarityFactory, TestLMJelinekMercerSimilarityFactory, TestFastWriter, TestJmxMonitoredMap, TimeZoneUtilsTest, TestFastOutputStream, ScriptEngineTest, OpenExchangeRatesOrgProviderTest, URLClassifyProcessorTest, PluginInfoTest, PreAnalyzedFieldTest, DateFieldTest, SpellingQueryConverterTest, DOMUtilTest, ClusterStateTest, TestDocumentBuilder, ZkNodePropsTest, SliceStateTest, SystemInfoHandlerTest, DistributedMLTComponentTest, TestRTGBase, SolrTestCaseJ4Test, TestCrossCoreJoin, TestCursorMarkWithoutUniqueKey, TestHighlightDedupGrouping, ActionThrottleTest, AssignTest, AsyncCallRequestStatusResponseTest, AsyncMigrateRouteKeyTest, CustomCollectionTest, LeaderInitiatedRecoveryOnShardRestartTest, MigrateRouteKeyTest, OverriddenZkACLAndCredentialsProvidersTest, OverseerCollectionConfigSetProcessorTest, OverseerRolesTest, SharedFSAutoReplicaFailoverTest, SharedFSAutoReplicaFailoverUtilsTest, SimpleCollectionCreateDeleteTest, TestConfigSetsAPI, TestConfigSetsAPIZkFailure, TestCryptoKeys, TestDistribDocBasedVersion, TestDownShardTolerantSearch, TestLeaderElectionZkExpiry, TestMiniSolrCloudCluster, TestRequestStatusCollectionAPI, HdfsBasicDistributedZk2Test]
   [junit4] Completed [445/547] on J1 in 111.54s, 1 test, 1 error <<< FAILURES!

[...truncated 355 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 547 suites (7 ignored), 2163 tests, 1 error, 32 ignored (18 assumptions) [seed: D9C7AF4E9AE9B76C]

Total time: 207 minutes 28 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1017 - Still Failing

[Apache Jenkins Server <je...@builds.apache.org>]

Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1017/

1 tests failed.
FAILED:  org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.test

Error Message:
Error from server at http://127.0.0.1:36518/awholynewcollection_4: non ok status: 500, message:Server Error

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:36518/awholynewcollection_4: non ok status: 500, message:Server Error
	at __randomizedtesting.SeedInfo.seed([F58C68861D2356FA:7DD8575CB3DF3B02]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:508)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:150)
	at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:943)
	at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:958)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.waitForNon403or404or503(AbstractFullDistribZkTestBase.java:1753)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testCollectionsAPI(CollectionsAPIDistributedZkTest.java:653)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test(CollectionsAPIDistributedZkTest.java:155)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1660)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:866)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:902)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:916)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:963)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:938)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:875)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:777)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:811)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:822)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 10975 lines...]
   [junit4] JVM J0: stdout was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J0-20151114_234001_339.sysout
   [junit4] >>> JVM J0: stdout (verbatim) ----
   [junit4] java.lang.OutOfMemoryError: GC overhead limit exceeded
   [junit4] Dumping heap to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/heapdumps/java_pid19078.hprof ...
   [junit4] Heap dump file created [638939390 bytes in 9.071 secs]
   [junit4] <<< JVM J0: EOF ----

   [junit4] JVM J0: stderr was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J0-20151114_234001_339.syserr
   [junit4] >>> JVM J0: stderr (verbatim) ----
   [junit4] WARN: Unhandled exception in event serialization. -> java.lang.OutOfMemoryError: GC overhead limit exceeded
   [junit4] <<< JVM J0: EOF ----

[...truncated 559 lines...]
   [junit4] Suite: org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F58C68861D2356FA-001/init-core-data-001
   [junit4]   2> 4197256 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F58C68861D2356FA]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false)
   [junit4]   2> 4197256 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F58C68861D2356FA]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   1> Formatting using clusterid: testClusterID
   [junit4]   2> 4197299 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F58C68861D2356FA]-worker) [    ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
   [junit4]   2> 4197312 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F58C68861D2356FA]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 4197328 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F58C68861D2356FA]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 4197361 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F58C68861D2356FA]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/hdfs to ./temp/Jetty_localhost_36591_hdfs____.3gewek/webapp
   [junit4]   2> 4197495 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F58C68861D2356FA]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 4197686 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F58C68861D2356FA]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:36591
   [junit4]   2> 4197796 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F58C68861D2356FA]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 4197798 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F58C68861D2356FA]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 4197811 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F58C68861D2356FA]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_60481_datanode____euxriy/webapp
   [junit4]   2> 4197938 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F58C68861D2356FA]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 4198098 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F58C68861D2356FA]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:60481
   [junit4]   2> 4198139 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F58C68861D2356FA]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 4198142 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F58C68861D2356FA]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 4198158 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F58C68861D2356FA]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_54118_datanode____.khzyps/webapp
   [junit4]   2> 4198205 INFO  (IPC Server handler 3 on 60351) [    ] BlockStateChange BLOCK* processReport: from storage DS-81243c2d-0bf3-43b9-99a2-f18d7d5a365a node DatanodeRegistration(127.0.0.1, datanodeUuid=ebe56e53-09d0-4832-adb4-537437018b20, infoPort=60481, ipcPort=45674, storageInfo=lv=-56;cid=testClusterID;nsid=1991185951;c=0), blocks: 0, hasStaleStorages: true, processing time: 0 msecs
   [junit4]   2> 4198205 INFO  (IPC Server handler 3 on 60351) [    ] BlockStateChange BLOCK* processReport: from storage DS-8e9d3920-f7ea-4637-b892-ae47e404abe9 node DatanodeRegistration(127.0.0.1, datanodeUuid=ebe56e53-09d0-4832-adb4-537437018b20, infoPort=60481, ipcPort=45674, storageInfo=lv=-56;cid=testClusterID;nsid=1991185951;c=0), blocks: 0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 4198297 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F58C68861D2356FA]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 4198454 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F58C68861D2356FA]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:54118
   [junit4]   2> 4198542 INFO  (IPC Server handler 8 on 60351) [    ] BlockStateChange BLOCK* processReport: from storage DS-a7faa45b-4eaa-482f-a43c-18ad15c2b0c5 node DatanodeRegistration(127.0.0.1, datanodeUuid=08a8b00c-65d5-4071-b04e-74f8f22d9d8e, infoPort=54118, ipcPort=42814, storageInfo=lv=-56;cid=testClusterID;nsid=1991185951;c=0), blocks: 0, hasStaleStorages: true, processing time: 0 msecs
   [junit4]   2> 4198543 INFO  (IPC Server handler 8 on 60351) [    ] BlockStateChange BLOCK* processReport: from storage DS-3ddf1adc-19f5-45e5-a5bd-7785777750ad node DatanodeRegistration(127.0.0.1, datanodeUuid=08a8b00c-65d5-4071-b04e-74f8f22d9d8e, infoPort=54118, ipcPort=42814, storageInfo=lv=-56;cid=testClusterID;nsid=1991185951;c=0), blocks: 0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 4198646 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 4198646 INFO  (Thread-59699) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 4198646 INFO  (Thread-59699) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 4198746 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.ZkTestServer start zk server on port:42590
   [junit4]   2> 4198746 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 4198747 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 4198750 INFO  (zkCallback-2566-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@3e2f9be5 name:ZooKeeperConnection Watcher:127.0.0.1:42590 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 4198750 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 4198750 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 4198750 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 4198753 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 4198754 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 4198755 INFO  (zkCallback-2567-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2da50c8c name:ZooKeeperConnection Watcher:127.0.0.1:42590/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 4198755 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 4198755 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 4198755 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 4198757 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 4198758 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 4198760 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 4198761 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 4198761 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 4198764 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 4198764 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 4198766 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 4198766 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 4198767 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 4198768 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 4198769 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 4198769 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 4198771 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 4198771 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 4198773 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 4198773 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 4198774 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 4198775 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 4198776 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 4198776 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 4198778 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 4198778 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 4198780 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 4198780 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 4199075 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F58C68861D2356FA-001/control-001/cores/collection1
   [junit4]   2> 4199077 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 4199079 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@4459fa43{/,null,AVAILABLE}
   [junit4]   2> 4199079 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.e.j.s.ServerConnector Started ServerConnector@28676c43{HTTP/1.1}{127.0.0.1:36938}
   [junit4]   2> 4199079 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.e.j.s.Server Started @4201821ms
   [junit4]   2> 4199079 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostPort=36938, solr.data.dir=hdfs://localhost:60351/hdfs__localhost_60351__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-5.x_solr_build_solr-core_test_J2_temp_solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F58C68861D2356FA-001_tempDir-002_control_data, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F58C68861D2356FA-001/control-001/cores, hostContext=/}
   [junit4]   2> 4199079 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@7b3cb2c6
   [junit4]   2> 4199080 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F58C68861D2356FA-001/control-001/'
   [junit4]   2> 4199080 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 4199080 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 4199096 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 4199097 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 4199098 INFO  (zkCallback-2568-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5d42ebd0 name:ZooKeeperConnection Watcher:127.0.0.1:42590/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 4199098 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 4199099 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 4199100 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 4199100 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F58C68861D2356FA-001/control-001/solr.xml
   [junit4]   2> 4199114 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F58C68861D2356FA-001/control-001/cores
   [junit4]   2> 4199114 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.CoreContainer New CoreContainer 799937137
   [junit4]   2> 4199114 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F58C68861D2356FA-001/control-001/]
   [junit4]   2> 4199115 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F58C68861D2356FA-001/control-001/lib
   [junit4]   2> 4199115 WARN  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F58C68861D2356FA-001/control-001/lib).
   [junit4]   2> 4199123 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 4199124 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 4199124 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 4199124 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 4199124 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:42590/solr
   [junit4]   2> 4199124 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 4199125 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 4199125 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 4199127 INFO  (zkCallback-2570-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7c3d3caa name:ZooKeeperConnection Watcher:127.0.0.1:42590 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 4199127 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 4199127 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 4199128 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [n:127.0.0.1:36938_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 4199129 INFO  (zkCallback-2571-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@14c02db4 name:ZooKeeperConnection Watcher:127.0.0.1:42590/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 4199130 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [n:127.0.0.1:36938_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 4199131 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [n:127.0.0.1:36938_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 4199133 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [n:127.0.0.1:36938_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 4199136 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [n:127.0.0.1:36938_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 4199138 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [n:127.0.0.1:36938_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 4199140 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [n:127.0.0.1:36938_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 4199142 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [n:127.0.0.1:36938_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 4199144 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [n:127.0.0.1:36938_    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 4199145 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [n:127.0.0.1:36938_    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 4199147 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [n:127.0.0.1:36938_    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 4199148 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [n:127.0.0.1:36938_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:36938_
   [junit4]   2> 4199149 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [n:127.0.0.1:36938_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:36938_
   [junit4]   2> 4199150 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [n:127.0.0.1:36938_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 4199152 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [n:127.0.0.1:36938_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 4199153 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [n:127.0.0.1:36938_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 4199154 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [n:127.0.0.1:36938_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94866545235394564-127.0.0.1:36938_-n_0000000000
   [junit4]   2> 4199154 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [n:127.0.0.1:36938_    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:36938_
   [junit4]   2> 4199154 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [n:127.0.0.1:36938_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 4199156 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [n:127.0.0.1:36938_    ] o.a.s.c.Overseer Overseer (id=94866545235394564-127.0.0.1:36938_-n_0000000000) starting
   [junit4]   2> 4199158 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [n:127.0.0.1:36938_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 4199166 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [n:127.0.0.1:36938_    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 4199167 INFO  (OverseerCollectionConfigSetProcessor-94866545235394564-127.0.0.1:36938_-n_0000000000) [n:127.0.0.1:36938_    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 4199167 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [n:127.0.0.1:36938_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 4199167 INFO  (OverseerStateUpdate-94866545235394564-127.0.0.1:36938_-n_0000000000) [n:127.0.0.1:36938_    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 4199183 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [n:127.0.0.1:36938_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 4199183 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [n:127.0.0.1:36938_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 4199184 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [n:127.0.0.1:36938_    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F58C68861D2356FA-001/control-001/cores
   [junit4]   2> 4199185 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [n:127.0.0.1:36938_    ] o.a.s.c.SolrCore Created CoreDescriptor: {transient=false, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F58C68861D2356FA-001/control-001/cores/collection1, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F58C68861D2356FA-001/control-001/cores/collection1/, loadOnStartup=true, coreNodeName=, dataDir=data/, config=solrconfig.xml, shard=, schema=schema.xml, name=collection1, configSetProperties=configsetprops.json, collection=control_collection}
   [junit4]   2> 4199186 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [n:127.0.0.1:36938_    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F58C68861D2356FA-001/control-001/cores/collection1/
   [junit4]   2> 4199186 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [n:127.0.0.1:36938_    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 4199187 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [n:127.0.0.1:36938_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 4199187 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [n:127.0.0.1:36938_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 4199187 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 4199187 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 4199190 INFO  (OverseerStateUpdate-94866545235394564-127.0.0.1:36938_-n_0000000000) [n:127.0.0.1:36938_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "node_name":"127.0.0.1:36938_",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "base_url":"http://127.0.0.1:36938",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "roles":null,
   [junit4]   2>   "collection":"control_collection"} current state version: 0
   [junit4]   2> 4199190 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 4199190 INFO  (OverseerStateUpdate-94866545235394564-127.0.0.1:36938_-n_0000000000) [n:127.0.0.1:36938_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "node_name":"127.0.0.1:36938_",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "base_url":"http://127.0.0.1:36938",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "roles":null,
   [junit4]   2>   "collection":"control_collection"}
   [junit4]   2> 4199191 INFO  (OverseerStateUpdate-94866545235394564-127.0.0.1:36938_-n_0000000000) [n:127.0.0.1:36938_    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 4199191 INFO  (OverseerStateUpdate-94866545235394564-127.0.0.1:36938_-n_0000000000) [n:127.0.0.1:36938_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 4199193 INFO  (zkCallback-2571-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 4200191 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 4200191 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 4200191 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 4200191 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 4200192 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 4200192 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F58C68861D2356FA-001/control-001/cores/collection1/'
   [junit4]   2> 4200193 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 4200193 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 4200210 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 4200217 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 4200284 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 4200313 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 4200315 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 4200321 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 4200583 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 4200591 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 4200593 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 4200600 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 4200603 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 4200606 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 4200606 WARN  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 4200606 WARN  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 4200606 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 4200607 WARN  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 4200607 WARN  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 4200608 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 4200608 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 4200608 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.HdfsDirectoryFactory
   [junit4]   2> 4200609 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:60351/solr_hdfs_home
   [junit4]   2> 4200609 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
   [junit4]   2> 4200609 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F58C68861D2356FA-001/control-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 4200609 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@7735b56d
   [junit4]   2> 4200609 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:60351/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 4200627 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:60351/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 4200628 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=hdfs://localhost:60351/solr_hdfs_home/control_collection/core_node1/data/index/
   [junit4]   2> 4200639 WARN  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory 'hdfs:/localhost:60351/solr_hdfs_home/control_collection/core_node1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 4200639 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:60351/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 4200686 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:60351/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 4200686 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=10, maxMergeAtOnceExplicit=11, maxMergedSegmentMB=0.5712890625, floorSegmentMB=0.4052734375, forceMergeDeletesPctAllowed=18.11247255870828, segmentsPerTier=46.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
   [junit4]   2> 4200721 INFO  (IPC Server handler 7 on 60351) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:52142 is added to blk_1073741825_1001{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-81243c2d-0bf3-43b9-99a2-f18d7d5a365a:NORMAL:127.0.0.1:33094|RBW], ReplicaUnderConstruction[[DISK]DS-3ddf1adc-19f5-45e5-a5bd-7785777750ad:NORMAL:127.0.0.1:52142|RBW]]} size 0
   [junit4]   2> 4200723 INFO  (IPC Server handler 8 on 60351) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:33094 is added to blk_1073741825_1001{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-81243c2d-0bf3-43b9-99a2-f18d7d5a365a:NORMAL:127.0.0.1:33094|RBW], ReplicaUnderConstruction[[DISK]DS-3ddf1adc-19f5-45e5-a5bd-7785777750ad:NORMAL:127.0.0.1:52142|RBW]]} size 0
   [junit4]   2> 4200726 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(HdfsDirectory@b92c1d99 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@681ee545; maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 4200726 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 4200728 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in hdfs://localhost:60351/solr_hdfs_home/control_collection/core_node1/data/
   [junit4]   2> 4200732 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 4200732 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 4200733 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 4200733 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 4200733 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 4200733 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 4200734 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 4200734 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 4200734 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 4200734 WARN  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
   [junit4]   2> 4200737 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 4200738 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 4200739 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 4200741 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 4200745 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: standard,/admin/properties,/get,/admin/logging,/admin/system,/update/json,/admin/segments,/admin/mbeans,/update/json/docs,/admin/file,/update,/admin/ping,/dump,/schema,/admin/threads,/replication,/admin/plugins,/config,/update/csv,/admin/luke
   [junit4]   2> 4200747 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 4200747 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
   [junit4]   2> 4200747 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing HdfsUpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 tlogDfsReplication=2
   [junit4]   2> 4200760 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 4200760 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 4200761 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=9, maxMergeAtOnceExplicit=4, maxMergedSegmentMB=49.2177734375, floorSegmentMB=1.4482421875, forceMergeDeletesPctAllowed=2.931814510927212, segmentsPerTier=24.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.10836246229761147
   [junit4]   2> 4200771 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(HdfsDirectory@b92c1d99 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@681ee545; maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 4200771 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 4200771 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@2300b23b[collection1] main
   [junit4]   2> 4200771 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 4200772 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 4200772 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 4200772 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 4200772 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 4200773 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 4200773 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 4200773 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 4200773 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 4200774 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 4200775 INFO  (searcherExecutor-8521-thread-1-processing-x:collection1 n:127.0.0.1:36938_ s:shard1 r:core_node1 c:control_collection) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@2300b23b[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 4200775 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 4200775 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 256 version buckets from index
   [junit4]   2> 4200775 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 4200775 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1517864725996634112
   [junit4]   2> 4200775 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 0.0ms to seed version buckets with highest version 1517864725996634112
   [junit4]   2> 4200775 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 4200776 INFO  (coreLoadExecutor-8520-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 4200776 INFO  (coreZkRegister-8514-thread-1-processing-x:collection1 n:127.0.0.1:36938_ s:shard1 r:core_node1 c:control_collection) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:36938 collection:control_collection shard:shard1
   [junit4]   2> 4200777 INFO  (coreZkRegister-8514-thread-1-processing-x:collection1 n:127.0.0.1:36938_ s:shard1 r:core_node1 c:control_collection) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 4200781 INFO  (coreZkRegister-8514-thread-1-processing-x:collection1 n:127.0.0.1:36938_ s:shard1 r:core_node1 c:control_collection) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94866545235394564-core_node1-n_0000000000
   [junit4]   2> 4200781 INFO  (coreZkRegister-8514-thread-1-processing-x:collection1 n:127.0.0.1:36938_ s:shard1 r:core_node1 c:control_collection) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 4200782 INFO  (coreZkRegister-8514-thread-1-processing-x:collection1 n:127.0.0.1:36938_ s:shard1 r:core_node1 c:control_collection) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 4200783 INFO  (coreZkRegister-8514-thread-1-processing-x:collection1 n:127.0.0.1:36938_ s:shard1 r:core_node1 c:control_collection) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 4200783 INFO  (coreZkRegister-8514-thread-1-processing-x:collection1 n:127.0.0.1:36938_ s:shard1 r:core_node1 c:control_collection) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:36938/collection1/
   [junit4]   2> 4200783 INFO  (OverseerStateUpdate-94866545235394564-127.0.0.1:36938_-n_0000000000) [n:127.0.0.1:36938_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 4200783 INFO  (coreZkRegister-8514-thread-1-processing-x:collection1 n:127.0.0.1:36938_ s:shard1 r:core_node1 c:control_collection) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 4200783 INFO  (coreZkRegister-8514-thread-1-processing-x:collection1 n:127.0.0.1:36938_ s:shard1 r:core_node1 c:control_collection) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:36938/collection1/ has no replicas
   [junit4]   2> 4200784 INFO  (zkCallback-2571-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 4200784 INFO  (coreZkRegister-8514-thread-1-processing-x:collection1 n:127.0.0.1:36938_ s:shard1 r:core_node1 c:control_collection) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 4200786 INFO  (coreZkRegister-8514-thread-1-processing-x:collection1 n:127.0.0.1:36938_ s:shard1 r:core_node1 c:control_collection) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 4200788 INFO  (coreZkRegister-8514-thread-1-processing-x:collection1 n:127.0.0.1:36938_ s:shard1 r:core_node1 c:control_collection) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:36938/collection1/ shard1
   [junit4]   2> 4200789 INFO  (OverseerStateUpdate-94866545235394564-127.0.0.1:36938_-n_0000000000) [n:127.0.0.1:36938_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:36938",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 2
   [junit4]   2> 4200891 INFO  (zkCallback-2571-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 4200939 INFO  (coreZkRegister-8514-thread-1-processing-x:collection1 n:127.0.0.1:36938_ s:shard1 r:core_node1 c:control_collection) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:36938/collection1/ and leader is http://127.0.0.1:36938/collection1/
   [junit4]   2> 4200939 INFO  (coreZkRegister-8514-thread-1-processing-x:collection1 n:127.0.0.1:36938_ s:shard1 r:core_node1 c:control_collection) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:36938
   [junit4]   2> 4200940 INFO  (coreZkRegister-8514-thread-1-processing-x:collection1 n:127.0.0.1:36938_ s:shard1 r:core_node1 c:control_collection) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 4200940 INFO  (coreZkRegister-8514-thread-1-processing-x:collection1 n:127.0.0.1:36938_ s:shard1 r:core_node1 c:control_collection) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 4200940 INFO  (coreZkRegister-8514-thread-1-processing-x:collection1 n:127.0.0.1:36938_ s:shard1 r:core_node1 c:control_collection) [n:127.0.0.1:36938_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 4200942 INFO  (OverseerStateUpdate-94866545235394564-127.0.0.1:36938_-n_0000000000) [n:127.0.0.1:36938_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "node_name":"127.0.0.1:36938_",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "base_url":"http://127.0.0.1:36938",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "collection":"control_collection"} current state version: 3
   [junit4]   2> 4200942 INFO  (OverseerStateUpdate-94866545235394564-127.0.0.1:36938_-n_0000000000) [n:127.0.0.1:36938_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "node_name":"127.0.0.1:36938_",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "base_url":"http://127.0.0.1:36938",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "collection":"control_collection"}
   [junit4]   2> 4201044 INFO  (zkCallback-2571-thread-1-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 4201189 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 4201190 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 4201191 INFO  (zkCallback-2573-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1a659205 name:ZooKeeperConnection Watcher:127.0.0.1:42590/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 4201191 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 4201191 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 4201192 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 4201194 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 4201194 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with stateFormat=2
   [junit4]   2> 4201194 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 4201195 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 4201196 INFO  (zkCallback-2574-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@be29249 name:ZooKeeperConnection Watcher:127.0.0.1:42590/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 4201196 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 4201196 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 4201198 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x151089ec9d30006, likely client has closed socket
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 4201198 INFO  (OverseerStateUpdate-94866545235394564-127.0.0.1:36938_-n_0000000000) [n:127.0.0.1:36938_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"create",
   [junit4]   2>   "name":"collection1",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "stateFormat":"2"} current state version: 4
   [junit4]   2> 4201199 INFO  (OverseerStateUpdate-94866545235394564-127.0.0.1:36938_-n_0000000000) [n:127.0.0.1:36938_    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 4201301 INFO  (OverseerStateUpdate-94866545235394564-127.0.0.1:36938_-n_0000000000) [n:127.0.0.1:36938_    ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/collection1/state.json
   [junit4]   2> 4201491 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F58C68861D2356FA-001/shard-1-001/cores/collection1
   [junit4]   2> 4201491 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F58C68861D2356FA-001/shard-1-001
   [junit4]   2> 4201492 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 4201494 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@35db059c{/,null,AVAILABLE}
   [junit4]   2> 4201495 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.e.j.s.ServerConnector Started ServerConnector@49f32389{HTTP/1.1}{127.0.0.1:36518}
   [junit4]   2> 4201495 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.e.j.s.Server Started @4204237ms
   [junit4]   2> 4201495 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F58C68861D2356FA-001/shard-1-001/cores, hostPort=36518, solrconfig=solrconfig.xml, solr.data.dir=hdfs://localhost:60351/hdfs__localhost_60351__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-5.x_solr_build_solr-core_test_J2_temp_solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F58C68861D2356FA-001_tempDir-002_jetty1, hostContext=/}
   [junit4]   2> 4201495 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@7b3cb2c6
   [junit4]   2> 4201495 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F58C68861D2356FA-001/shard-1-001/'
   [junit4]   2> 4201495 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 4201495 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 4201512 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 4201513 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 4201514 INFO  (zkCallback-2575-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5e711f45 name:ZooKeeperConnection Watcher:127.0.0.1:42590/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 4201514 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 4201514 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 4201515 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 4201515 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F58C68861D2356FA-001/shard-1-001/solr.xml
   [junit4]   2> 4201530 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F58C68861D2356FA-001/shard-1-001/cores
   [junit4]   2> 4201530 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.CoreContainer New CoreContainer 2136837154
   [junit4]   2> 4201530 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F58C68861D2356FA-001/shard-1-001/]
   [junit4]   2> 4201530 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F58C68861D2356FA-001/shard-1-001/lib
   [junit4]   2> 4201530 WARN  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F58C68861D2356FA-001/shard-1-001/lib).
   [junit4]   2> 4201538 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 4201539 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 4201540 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 4201540 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 4201540 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:42590/solr
   [junit4]   2> 4201540 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 4201540 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 4201541 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 4201543 INFO  (zkCallback-2577-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@54983e84 name:ZooKeeperConnection Watcher:127.0.0.1:42590 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 4201543 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 4201544 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 4201545 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [n:127.0.0.1:36518_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 4201547 INFO  (zkCallback-2578-thread-1-processing-n:127.0.0.1:36518_) [n:127.0.0.1:36518_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1be70ed5 name:ZooKeeperConnection Watcher:127.0.0.1:42590/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 4201548 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [n:127.0.0.1:36518_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 4201553 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [n:127.0.0.1:36518_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 4202559 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [n:127.0.0.1:36518_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:36518_
   [junit4]   2> 4202560 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [n:127.0.0.1:36518_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:36518_
   [junit4]   2> 4202561 INFO  (zkCallback-2571-thread-2-processing-n:127.0.0.1:36938_) [n:127.0.0.1:36938_    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 4202561 INFO  (zkCallback-2573-thread-1) [    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 4202561 INFO  (zkCallback-2578-thread-1-processing-n:127.0.0.1:36518_) [n:127.0.0.1:36518_    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 4202562 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [n:127.0.0.1:36518_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 4202563 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [n:127.0.0.1:36518_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94866545235394569-127.0.0.1:36518_-n_0000000001
   [junit4]   2> 4202564 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [n:127.0.0.1:36518_    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/94866545235394564-127.0.0.1:36938_-n_0000000000 to know if I could be the leader
   [junit4]   2> 4202579 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [n:127.0.0.1:36518_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 4202579 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [n:127.0.0.1:36518_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 4202581 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [n:127.0.0.1:36518_    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F58C68861D2356FA-001/shard-1-001/cores
   [junit4]   2> 4202582 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[F58C68861D2356FA]) [n:127.0.0.1:36518_    ] o.a.s.c.SolrCore Created CoreDescriptor: 

[...truncated too long message...]

in-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-authn-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-authz-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-changelog-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-collective-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-event-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-exception-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-journal-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-normalization-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-operational-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-referral-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-schema-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-subtree-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-trigger-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-jdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-jdbm1-2.0.0-M2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-kerberos-codec-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-ldif-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-mavibot-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-protocol-kerberos-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-protocol-ldap-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-protocol-shared-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-xdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/api-all-1.0.0-M20.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/bcprov-jdk15-1.45.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/cglib-nodep-2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/commons-collections-3.2.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/commons-math3-3.4.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/easymock-3.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/ehcache-core-2.4.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/hadoop-common-2.6.0-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/hadoop-hdfs-2.6.0-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/hadoop-minikdc-2.6.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jackson-annotations-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jackson-databind-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jcl-over-slf4j-1.7.7.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jersey-core-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jersey-server-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jetty-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jetty-util-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/mina-core-2.0.0-M5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/objenesis-1.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/analysis/icu/lucene-analyzers-icu-5.4.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-analysis-extras/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/contrib/analysis-extras/lib/icu4j-54.1.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-launcher.jar:/x1/jenkins/.ant/lib/ivy-2.3.0.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-antlr.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-regexp.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-resolver.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-net.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jmf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-oro.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-xalan2.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-logging.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jsch.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit4.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-javamail.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jai.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jdepend.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bsf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-swing.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-netrexx.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bcel.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-testutil.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-log4j.jar:/x1/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.7/lib/tools.jar:/x1/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.2.0.jar com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J0-20151114_234001_339.events @/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J0-20151114_234001_339.suites -stdin
   [junit4] ERROR: JVM J0 ended with an exception: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1503)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:130)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:963)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:960)
   [junit4] 	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4] 	at java.lang.Thread.run(Thread.java:745)

BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: At least one slave process threw an exception, first: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.

Total time: 219 minutes 36 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1016 - Still Failing

[Apache Jenkins Server <je...@builds.apache.org>]

Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1016/

1 tests failed.
FAILED:  org.apache.solr.cloud.hdfs.StressHdfsTest.test

Error Message:
Timeout occured while waiting response from server at: http://127.0.0.1:44745/bnc/delete_data_dir

Stack Trace:
org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:44745/bnc/delete_data_dir
	at __randomizedtesting.SeedInfo.seed([F14E7BEDE03608BB:791A44374ECA6543]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:585)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:150)
	at org.apache.solr.client.solrj.SolrClient.commit(SolrClient.java:483)
	at org.apache.solr.client.solrj.SolrClient.commit(SolrClient.java:464)
	at org.apache.solr.cloud.hdfs.StressHdfsTest.createAndDeleteCollection(StressHdfsTest.java:197)
	at org.apache.solr.cloud.hdfs.StressHdfsTest.test(StressHdfsTest.java:100)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1660)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:866)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:902)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:916)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:963)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:938)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:875)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:777)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:811)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:822)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.net.SocketTimeoutException: Read timed out
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.read(SocketInputStream.java:152)
	at java.net.SocketInputStream.read(SocketInputStream.java:122)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:160)
	at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:84)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:273)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
	at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
	at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283)
	at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251)
	at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197)
	at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)
	at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)
	at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685)
	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487)
	at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:480)
	... 53 more




Build Log:
[...truncated 10402 lines...]
   [junit4] Suite: org.apache.solr.cloud.hdfs.StressHdfsTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.StressHdfsTest_F14E7BEDE03608BB-001/init-core-data-001
   [junit4]   2> 491245 INFO  (SUITE-StressHdfsTest-seed#[F14E7BEDE03608BB]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /bnc/
   [junit4]   1> Formatting using clusterid: testClusterID
   [junit4]   2> 491318 WARN  (SUITE-StressHdfsTest-seed#[F14E7BEDE03608BB]-worker) [    ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
   [junit4]   2> 491328 WARN  (SUITE-StressHdfsTest-seed#[F14E7BEDE03608BB]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 491331 INFO  (SUITE-StressHdfsTest-seed#[F14E7BEDE03608BB]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 491354 INFO  (SUITE-StressHdfsTest-seed#[F14E7BEDE03608BB]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/hdfs to ./temp/Jetty_localhost_59930_hdfs____.57fuw4/webapp
   [junit4]   2> 491573 INFO  (SUITE-StressHdfsTest-seed#[F14E7BEDE03608BB]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 491955 INFO  (SUITE-StressHdfsTest-seed#[F14E7BEDE03608BB]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:59930
   [junit4]   2> 492198 WARN  (SUITE-StressHdfsTest-seed#[F14E7BEDE03608BB]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 492200 INFO  (SUITE-StressHdfsTest-seed#[F14E7BEDE03608BB]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 492221 INFO  (SUITE-StressHdfsTest-seed#[F14E7BEDE03608BB]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_38441_datanode____.fct78t/webapp
   [junit4]   2> 492492 INFO  (SUITE-StressHdfsTest-seed#[F14E7BEDE03608BB]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 492814 INFO  (SUITE-StressHdfsTest-seed#[F14E7BEDE03608BB]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:38441
   [junit4]   2> 493050 WARN  (SUITE-StressHdfsTest-seed#[F14E7BEDE03608BB]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 493052 INFO  (SUITE-StressHdfsTest-seed#[F14E7BEDE03608BB]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 493126 INFO  (SUITE-StressHdfsTest-seed#[F14E7BEDE03608BB]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_59637_datanode____.kyfdwj/webapp
   [junit4]   2> 493181 INFO  (IPC Server handler 2 on 49818) [    ] BlockStateChange BLOCK* processReport: from storage DS-847ac855-bf43-4298-9ab8-a2efa42ad71a node DatanodeRegistration(127.0.0.1, datanodeUuid=417cd5b5-071e-4cd0-b33b-3a59817f4a13, infoPort=38441, ipcPort=46888, storageInfo=lv=-56;cid=testClusterID;nsid=819706740;c=0), blocks: 0, hasStaleStorages: true, processing time: 0 msecs
   [junit4]   2> 493182 INFO  (IPC Server handler 2 on 49818) [    ] BlockStateChange BLOCK* processReport: from storage DS-cc194b65-4d12-47e5-8dff-00028b0d85c2 node DatanodeRegistration(127.0.0.1, datanodeUuid=417cd5b5-071e-4cd0-b33b-3a59817f4a13, infoPort=38441, ipcPort=46888, storageInfo=lv=-56;cid=testClusterID;nsid=819706740;c=0), blocks: 0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 493361 INFO  (SUITE-StressHdfsTest-seed#[F14E7BEDE03608BB]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 493731 INFO  (SUITE-StressHdfsTest-seed#[F14E7BEDE03608BB]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:59637
   [junit4]   2> 494005 INFO  (IPC Server handler 8 on 49818) [    ] BlockStateChange BLOCK* processReport: from storage DS-d65f7d23-1798-4bb7-8e48-63fb5dfcaaf2 node DatanodeRegistration(127.0.0.1, datanodeUuid=e51316a5-a6c4-4a5f-9ab2-3e9dbeb260c1, infoPort=59637, ipcPort=54984, storageInfo=lv=-56;cid=testClusterID;nsid=819706740;c=0), blocks: 0, hasStaleStorages: true, processing time: 0 msecs
   [junit4]   2> 494005 INFO  (IPC Server handler 8 on 49818) [    ] BlockStateChange BLOCK* processReport: from storage DS-a6552db6-73e4-4410-a10b-c8a5ba21e554 node DatanodeRegistration(127.0.0.1, datanodeUuid=e51316a5-a6c4-4a5f-9ab2-3e9dbeb260c1, infoPort=59637, ipcPort=54984, storageInfo=lv=-56;cid=testClusterID;nsid=819706740;c=0), blocks: 0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 494147 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 494150 INFO  (Thread-1847) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 494150 INFO  (Thread-1847) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 494250 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.ZkTestServer start zk server on port:45773
   [junit4]   2> 494250 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 494257 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 494276 INFO  (zkCallback-144-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7d0526f3 name:ZooKeeperConnection Watcher:127.0.0.1:45773 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 494276 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 494276 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 494276 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 494279 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 494282 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 494298 INFO  (zkCallback-145-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7465aad4 name:ZooKeeperConnection Watcher:127.0.0.1:45773/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 494298 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 494298 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 494299 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 494301 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 494302 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 494303 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 494304 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 494304 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 494306 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 494307 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 494308 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 494308 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 494309 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 494310 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 494311 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 494311 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 494312 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 494312 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 494314 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 494314 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 494315 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 494315 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 494316 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 494316 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 494318 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 494318 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 494319 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 494319 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 495031 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.StressHdfsTest_F14E7BEDE03608BB-001/control-001/cores/collection1
   [junit4]   2> 495033 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 495035 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@2d2987df{/bnc,null,AVAILABLE}
   [junit4]   2> 495036 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.e.j.s.ServerConnector Started ServerConnector@773c3625{HTTP/1.1}{127.0.0.1:39842}
   [junit4]   2> 495036 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.e.j.s.Server Started @498068ms
   [junit4]   2> 495036 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.StressHdfsTest_F14E7BEDE03608BB-001/control-001/cores, hostContext=/bnc, hostPort=39842, solr.data.dir=hdfs://localhost:49818/hdfs__localhost_49818__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-5.x_solr_build_solr-core_test_J0_temp_solr.cloud.hdfs.StressHdfsTest_F14E7BEDE03608BB-001_tempDir-002_control_data}
   [junit4]   2> 495036 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@7b3cb2c6
   [junit4]   2> 495036 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.StressHdfsTest_F14E7BEDE03608BB-001/control-001/'
   [junit4]   2> 495036 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 495037 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 495103 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 495105 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 495116 INFO  (zkCallback-146-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@a531ba0 name:ZooKeeperConnection Watcher:127.0.0.1:45773/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 495117 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 495117 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 495120 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 495120 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.StressHdfsTest_F14E7BEDE03608BB-001/control-001/solr.xml
   [junit4]   2> 495141 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.StressHdfsTest_F14E7BEDE03608BB-001/control-001/cores
   [junit4]   2> 495141 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.CoreContainer New CoreContainer 1574675448
   [junit4]   2> 495141 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.StressHdfsTest_F14E7BEDE03608BB-001/control-001/]
   [junit4]   2> 495141 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.StressHdfsTest_F14E7BEDE03608BB-001/control-001/lib
   [junit4]   2> 495141 WARN  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.StressHdfsTest_F14E7BEDE03608BB-001/control-001/lib).
   [junit4]   2> 495150 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 495151 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 495152 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 495152 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 495152 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:45773/solr
   [junit4]   2> 495152 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 495152 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 495152 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 495154 INFO  (zkCallback-148-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1eb1f18e name:ZooKeeperConnection Watcher:127.0.0.1:45773 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 495154 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 495155 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 495157 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [n:127.0.0.1:39842_bnc    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 495165 INFO  (zkCallback-149-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6737db10 name:ZooKeeperConnection Watcher:127.0.0.1:45773/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 495165 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [n:127.0.0.1:39842_bnc    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 495172 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [n:127.0.0.1:39842_bnc    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 495181 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [n:127.0.0.1:39842_bnc    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 495189 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [n:127.0.0.1:39842_bnc    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 495195 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [n:127.0.0.1:39842_bnc    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 495203 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [n:127.0.0.1:39842_bnc    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 495207 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [n:127.0.0.1:39842_bnc    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 495216 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [n:127.0.0.1:39842_bnc    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 495226 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [n:127.0.0.1:39842_bnc    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 495230 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [n:127.0.0.1:39842_bnc    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 495232 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [n:127.0.0.1:39842_bnc    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:39842_bnc
   [junit4]   2> 495232 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [n:127.0.0.1:39842_bnc    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:39842_bnc
   [junit4]   2> 495234 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [n:127.0.0.1:39842_bnc    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 495235 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [n:127.0.0.1:39842_bnc    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 495237 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [n:127.0.0.1:39842_bnc    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 495238 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [n:127.0.0.1:39842_bnc    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94861200246046724-127.0.0.1:39842_bnc-n_0000000000
   [junit4]   2> 495238 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [n:127.0.0.1:39842_bnc    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:39842_bnc
   [junit4]   2> 495238 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [n:127.0.0.1:39842_bnc    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 495241 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [n:127.0.0.1:39842_bnc    ] o.a.s.c.Overseer Overseer (id=94861200246046724-127.0.0.1:39842_bnc-n_0000000000) starting
   [junit4]   2> 495243 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [n:127.0.0.1:39842_bnc    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 495248 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [n:127.0.0.1:39842_bnc    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 495249 INFO  (OverseerCollectionConfigSetProcessor-94861200246046724-127.0.0.1:39842_bnc-n_0000000000) [n:127.0.0.1:39842_bnc    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 495249 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [n:127.0.0.1:39842_bnc    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 495249 INFO  (OverseerStateUpdate-94861200246046724-127.0.0.1:39842_bnc-n_0000000000) [n:127.0.0.1:39842_bnc    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 495272 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [n:127.0.0.1:39842_bnc    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 495272 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [n:127.0.0.1:39842_bnc    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 495273 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [n:127.0.0.1:39842_bnc    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.StressHdfsTest_F14E7BEDE03608BB-001/control-001/cores
   [junit4]   2> 495274 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [n:127.0.0.1:39842_bnc    ] o.a.s.c.SolrCore Created CoreDescriptor: {config=solrconfig.xml, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.StressHdfsTest_F14E7BEDE03608BB-001/control-001/cores/collection1, coreNodeName=, name=collection1, transient=false, collection=control_collection, dataDir=data/, schema=schema.xml, loadOnStartup=true, shard=, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.StressHdfsTest_F14E7BEDE03608BB-001/control-001/cores/collection1/, configSetProperties=configsetprops.json}
   [junit4]   2> 495274 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [n:127.0.0.1:39842_bnc    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.StressHdfsTest_F14E7BEDE03608BB-001/control-001/cores/collection1/
   [junit4]   2> 495275 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [n:127.0.0.1:39842_bnc    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 495276 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [n:127.0.0.1:39842_bnc    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0
   [junit4]   2> 495276 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [n:127.0.0.1:39842_bnc    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 495280 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 495280 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 495285 INFO  (OverseerStateUpdate-94861200246046724-127.0.0.1:39842_bnc-n_0000000000) [n:127.0.0.1:39842_bnc    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "roles":null,
   [junit4]   2>   "node_name":"127.0.0.1:39842_bnc",
   [junit4]   2>   "base_url":"http://127.0.0.1:39842/bnc",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 495285 INFO  (OverseerStateUpdate-94861200246046724-127.0.0.1:39842_bnc-n_0000000000) [n:127.0.0.1:39842_bnc    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "roles":null,
   [junit4]   2>   "node_name":"127.0.0.1:39842_bnc",
   [junit4]   2>   "base_url":"http://127.0.0.1:39842/bnc",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 495285 INFO  (OverseerStateUpdate-94861200246046724-127.0.0.1:39842_bnc-n_0000000000) [n:127.0.0.1:39842_bnc    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 495288 INFO  (OverseerStateUpdate-94861200246046724-127.0.0.1:39842_bnc-n_0000000000) [n:127.0.0.1:39842_bnc    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 495291 INFO  (zkCallback-149-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 495291 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 495291 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 495292 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 495292 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 495292 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 495293 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 495293 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.StressHdfsTest_F14E7BEDE03608BB-001/control-001/cores/collection1/'
   [junit4]   2> 495293 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 495294 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 495314 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 495322 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 495337 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 495380 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 495382 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 495393 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 496370 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 496390 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 496391 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 496426 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 496440 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 496442 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 496443 WARN  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 496443 WARN  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 496443 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 496444 WARN  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 496444 WARN  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 496445 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0
   [junit4]   2> 496445 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 496445 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.HdfsDirectoryFactory
   [junit4]   2> 496445 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:49818/solr_hdfs_home
   [junit4]   2> 496445 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
   [junit4]   2> 496445 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.StressHdfsTest_F14E7BEDE03608BB-001/control-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 496446 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1c1dc16a
   [junit4]   2> 496446 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:49818/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 496479 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:49818/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 496480 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=hdfs://localhost:49818/solr_hdfs_home/control_collection/core_node1/data/index/
   [junit4]   2> 496500 WARN  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory 'hdfs:/localhost:49818/solr_hdfs_home/control_collection/core_node1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 496502 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:49818/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 496527 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
   [junit4]   2> 496527 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
   [junit4]   2> 496527 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
   [junit4]   2> 496560 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
   [junit4]   2> 496561 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:49818/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 496561 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=632344122, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
   [junit4]   2> 496634 INFO  (IPC Server handler 8 on 49818) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:59754 is added to blk_1073741825_1001{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-d65f7d23-1798-4bb7-8e48-63fb5dfcaaf2:NORMAL:127.0.0.1:38233|RBW], ReplicaUnderConstruction[[DISK]DS-847ac855-bf43-4298-9ab8-a2efa42ad71a:NORMAL:127.0.0.1:59754|RBW]]} size 0
   [junit4]   2> 496638 INFO  (IPC Server handler 9 on 49818) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:38233 is added to blk_1073741825_1001{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-d65f7d23-1798-4bb7-8e48-63fb5dfcaaf2:NORMAL:127.0.0.1:38233|RBW], ReplicaUnderConstruction[[DISK]DS-847ac855-bf43-4298-9ab8-a2efa42ad71a:NORMAL:127.0.0.1:59754|RBW]]} size 0
   [junit4]   2> 496646 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(BlockDirectory(HdfsDirectory@b7f2dee8 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@13d880a6); maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 496646 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 496659 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 496660 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 496660 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 496660 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 496660 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 496660 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 496661 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 496661 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 496661 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 496661 WARN  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
   [junit4]   2> 496664 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 496667 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 496668 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 496670 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 496672 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in hdfs://localhost:49818/solr_hdfs_home/control_collection/core_node1/data/
   [junit4]   2> 496692 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /update/json/docs,/update/json,/replication,/admin/system,/admin/luke,/config,/update/csv,/admin/segments,/admin/threads,/admin/plugins,/update,/admin/file,/admin/mbeans,/dump,/admin/logging,standard,/admin/properties,/schema,/get,/admin/ping
   [junit4]   2> 496694 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 496699 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
   [junit4]   2> 496699 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing HdfsUpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 tlogDfsReplication=2
   [junit4]   2> 496737 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 496737 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 496738 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=15, maxMergeAtOnceExplicit=28, maxMergedSegmentMB=0.8203125, floorSegmentMB=1.4462890625, forceMergeDeletesPctAllowed=14.568625339501043, segmentsPerTier=15.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.47258112760280724
   [junit4]   2> 496763 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(BlockDirectory(HdfsDirectory@b7f2dee8 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@13d880a6); maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 496763 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 496763 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@691f6645[collection1] main
   [junit4]   2> 496763 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 496764 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 496764 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 496765 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 496765 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 496765 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 496766 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 496766 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 496766 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 496766 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 496768 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 496768 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 256 version buckets from index
   [junit4]   2> 496768 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 496768 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1517779206673530880
   [junit4]   2> 496768 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 0.0ms to seed version buckets with highest version 1517779206673530880
   [junit4]   2> 496768 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 496769 INFO  (searcherExecutor-517-thread-1-processing-n:127.0.0.1:39842_bnc s:shard1 c:control_collection r:core_node1 x:collection1) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@691f6645[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 496770 INFO  (coreLoadExecutor-516-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 496778 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 496797 INFO  (coreZkRegister-510-thread-1-processing-n:127.0.0.1:39842_bnc s:shard1 c:control_collection r:core_node1 x:collection1) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:39842/bnc collection:control_collection shard:shard1
   [junit4]   2> 496800 INFO  (coreZkRegister-510-thread-1-processing-n:127.0.0.1:39842_bnc s:shard1 c:control_collection r:core_node1 x:collection1) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 496812 INFO  (coreZkRegister-510-thread-1-processing-n:127.0.0.1:39842_bnc s:shard1 c:control_collection r:core_node1 x:collection1) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94861200246046724-core_node1-n_0000000000
   [junit4]   2> 496812 INFO  (coreZkRegister-510-thread-1-processing-n:127.0.0.1:39842_bnc s:shard1 c:control_collection r:core_node1 x:collection1) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 496815 INFO  (OverseerStateUpdate-94861200246046724-127.0.0.1:39842_bnc-n_0000000000) [n:127.0.0.1:39842_bnc    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 496816 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 496816 INFO  (zkCallback-149-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 496819 INFO  (zkCallback-151-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@620a97ac name:ZooKeeperConnection Watcher:127.0.0.1:45773/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 496819 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 496819 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 496819 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 496834 INFO  (coreZkRegister-510-thread-1-processing-n:127.0.0.1:39842_bnc s:shard1 c:control_collection r:core_node1 x:collection1) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 496834 INFO  (coreZkRegister-510-thread-1-processing-n:127.0.0.1:39842_bnc s:shard1 c:control_collection r:core_node1 x:collection1) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 496834 INFO  (coreZkRegister-510-thread-1-processing-n:127.0.0.1:39842_bnc s:shard1 c:control_collection r:core_node1 x:collection1) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:39842/bnc/collection1/
   [junit4]   2> 496834 INFO  (coreZkRegister-510-thread-1-processing-n:127.0.0.1:39842_bnc s:shard1 c:control_collection r:core_node1 x:collection1) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 496834 INFO  (coreZkRegister-510-thread-1-processing-n:127.0.0.1:39842_bnc s:shard1 c:control_collection r:core_node1 x:collection1) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:39842/bnc/collection1/ has no replicas
   [junit4]   2> 496835 INFO  (coreZkRegister-510-thread-1-processing-n:127.0.0.1:39842_bnc s:shard1 c:control_collection r:core_node1 x:collection1) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 496835 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 496843 INFO  (coreZkRegister-510-thread-1-processing-n:127.0.0.1:39842_bnc s:shard1 c:control_collection r:core_node1 x:collection1) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 496845 INFO  (coreZkRegister-510-thread-1-processing-n:127.0.0.1:39842_bnc s:shard1 c:control_collection r:core_node1 x:collection1) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:39842/bnc/collection1/ shard1
   [junit4]   2> 496846 INFO  (OverseerStateUpdate-94861200246046724-127.0.0.1:39842_bnc-n_0000000000) [n:127.0.0.1:39842_bnc    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:39842/bnc",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 2
   [junit4]   2> 496948 INFO  (zkCallback-151-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 496948 INFO  (zkCallback-149-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 496983 INFO  (coreZkRegister-510-thread-1-processing-n:127.0.0.1:39842_bnc s:shard1 c:control_collection r:core_node1 x:collection1) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:39842/bnc/collection1/ and leader is http://127.0.0.1:39842/bnc/collection1/
   [junit4]   2> 496983 INFO  (coreZkRegister-510-thread-1-processing-n:127.0.0.1:39842_bnc s:shard1 c:control_collection r:core_node1 x:collection1) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:39842/bnc
   [junit4]   2> 496983 INFO  (coreZkRegister-510-thread-1-processing-n:127.0.0.1:39842_bnc s:shard1 c:control_collection r:core_node1 x:collection1) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 496984 INFO  (coreZkRegister-510-thread-1-processing-n:127.0.0.1:39842_bnc s:shard1 c:control_collection r:core_node1 x:collection1) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 496984 INFO  (coreZkRegister-510-thread-1-processing-n:127.0.0.1:39842_bnc s:shard1 c:control_collection r:core_node1 x:collection1) [n:127.0.0.1:39842_bnc c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 496985 INFO  (OverseerStateUpdate-94861200246046724-127.0.0.1:39842_bnc-n_0000000000) [n:127.0.0.1:39842_bnc    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "roles":null,
   [junit4]   2>   "node_name":"127.0.0.1:39842_bnc",
   [junit4]   2>   "base_url":"http://127.0.0.1:39842/bnc",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "operation":"state"} current state version: 3
   [junit4]   2> 496986 INFO  (OverseerStateUpdate-94861200246046724-127.0.0.1:39842_bnc-n_0000000000) [n:127.0.0.1:39842_bnc    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "roles":null,
   [junit4]   2>   "node_name":"127.0.0.1:39842_bnc",
   [junit4]   2>   "base_url":"http://127.0.0.1:39842/bnc",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 497089 INFO  (zkCallback-151-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 497089 INFO  (zkCallback-149-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 498391 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.StressHdfsTest_F14E7BEDE03608BB-001/shard-1-001/cores/collection1
   [junit4]   2> 498392 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.StressHdfsTest_F14E7BEDE03608BB-001/shard-1-001
   [junit4]   2> 498399 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 498475 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@333c109{/bnc,null,AVAILABLE}
   [junit4]   2> 498476 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.e.j.s.ServerConnector Started ServerConnector@2b66979a{HTTP/1.1}{127.0.0.1:52555}
   [junit4]   2> 498476 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.e.j.s.Server Started @501508ms
   [junit4]   2> 498476 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/bnc, solr.data.dir=hdfs://localhost:49818/hdfs__localhost_49818__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-5.x_solr_build_solr-core_test_J0_temp_solr.cloud.hdfs.StressHdfsTest_F14E7BEDE03608BB-001_tempDir-002_jetty1, hostPort=52555, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.StressHdfsTest_F14E7BEDE03608BB-001/shard-1-001/cores, solrconfig=solrconfig.xml}
   [junit4]   2> 498476 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@7b3cb2c6
   [junit4]   2> 498476 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.StressHdfsTest_F14E7BEDE03608BB-001/shard-1-001/'
   [junit4]   2> 498476 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 498476 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 498592 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 498615 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 498616 INFO  (zkCallback-152-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@3b033f94 name:ZooKeeperConnection Watcher:127.0.0.1:45773/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 498616 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 498616 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 498621 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 498621 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.StressHdfsTest_F14E7BEDE03608BB-001/shard-1-001/solr.xml
   [junit4]   2> 498654 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.StressHdfsTest_F14E7BEDE03608BB-001/shard-1-001/cores
   [junit4]   2> 498654 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.CoreContainer New CoreContainer 79431362
   [junit4]   2> 498654 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.StressHdfsTest_F14E7BEDE03608BB-001/shard-1-001/]
   [junit4]   2> 498655 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.StressHdfsTest_F14E7BEDE03608BB-001/shard-1-001/lib
   [junit4]   2> 498655 WARN  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.StressHdfsTest_F14E7BEDE03608BB-001/shard-1-001/lib).
   [junit4]   2> 498699 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 498700 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 498701 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 498701 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 498701 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:45773/solr
   [junit4]   2> 498701 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 498701 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 498776 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 498806 INFO  (zkCallback-154-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2f46703a name:ZooKeeperConnection Watcher:127.0.0.1:45773 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 498807 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 498807 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 498818 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [n:127.0.0.1:52555_bnc    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 498821 INFO  (zkCallback-155-thread-1-processing-n:127.0.0.1:52555_bnc) [n:127.0.0.1:52555_bnc    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6422400e name:ZooKeeperConnection Watcher:127.0.0.1:45773/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 498822 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [n:127.0.0.1:52555_bnc    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 498828 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [n:127.0.0.1:52555_bnc    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 499833 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [n:127.0.0.1:52555_bnc    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:52555_bnc
   [junit4]   2> 499834 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [n:127.0.0.1:52555_bnc    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:52555_bnc
   [junit4]   2> 499838 INFO  (zkCallback-149-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 499838 INFO  (zkCallback-151-thread-1) [    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 499839 INFO  (zkCallback-155-thread-1-processing-n:127.0.0.1:52555_bnc) [n:127.0.0.1:52555_bnc    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 499840 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [n:127.0.0.1:52555_bnc    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 499841 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [n:127.0.0.1:52555_bnc    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94861200246046728-127.0.0.1:52555_bnc-n_0000000001
   [junit4]   2> 499842 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [n:127.0.0.1:52555_bnc    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/94861200246046724-127.0.0.1:39842_bnc-n_0000000000 to know if I could be the leader
   [junit4]   2> 499893 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [n:127.0.0.1:52555_bnc    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 499893 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [n:127.0.0.1:52555_bnc    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 499894 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [n:127.0.0.1:52555_bnc    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.StressHdfsTest_F14E7BEDE03608BB-001/shard-1-001/cores
   [junit4]   2> 499895 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [n:127.0.0.1:52555_bnc    ] o.a.s.c.SolrCore Created CoreDescriptor: {schema=schema.xml, config=solrconfig.xml, shard=, loadOnStartup=true, coreNodeName=, dataDir=data/, collection=collection1, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.StressHdfsTest_F14E7BEDE03608BB-001/shard-1-001/cores/collection1/, configSetProperties=configsetprops.json, name=collection1, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.StressHdfsTest_F14E7BEDE03608BB-001/shard-1-001/cores/collection1, transient=false}
   [junit4]   2> 499895 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [n:127.0.0.1:52555_bnc    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.StressHdfsTest_F14E7BEDE03608BB-001/shard-1-001/cores/collection1/
   [junit4]   2> 499895 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [n:127.0.0.1:52555_bnc    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 499903 INFO  (coreLoadExecutor-527-thread-1-processing-n:127.0.0.1:52555_bnc) [n:127.0.0.1:52555_bnc c:collection1   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 499903 INFO  (coreLoadExecutor-527-thread-1-processing-n:127.0.0.1:52555_bnc) [n:127.0.0.1:52555_bnc c:collection1   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 499906 INFO  (coreLoadExecutor-527-thread-1-processing-n:127.0.0.1:52555_bnc) [n:127.0.0.1:52555_bnc c:collection1   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 499908 INFO  (OverseerStateUpdate-94861200246046724-127.0.0.1:39842_bnc-n_0000000000) [n:127.0.0.1:39842_bnc    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "roles":null,
   [junit4]   2>   "node_name":"127.0.0.1:52555_bnc",
   [junit4]   2>   "base_url":"http://127.0.0.1:52555/bnc",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "operation":"state"} current state version: 4
   [junit4]   2> 499909 INFO  (OverseerStateUpdate-94861200246046724-127.0.0.1:39842_bnc-n_0000000000) [n:127.0.0.1:39842_bnc    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "roles":null,
   [junit4]   2>   "node_name":"127.0.0.1:52555_bnc",
   [junit4]   2>   "base_url":"http://127.0.0.1:52555/bnc",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 499909 INFO  (OverseerStateUpdate-94861200246046724-127.0.0.1:39842_bnc-n_0000000000) [n:127.0.0.1:39842_bnc    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 499909 INFO  (OverseerStateUpdate-94861200246046724-127.0.0.1:39842_bnc-n_0000000000) [n:127.0.0.1:39842_bnc    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 499917 INFO  (zkCallback-151-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 499917 INFO  (zkCallback-155-thread-1-processing-n:127.0.0.1:52555_bnc) [n:127.0.0.1:52555_bnc    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 499918 INFO  (zkCallback-149-thread-1-processing-n:127.0.0.1:39842_bnc) [n:127.0.0.1:39842_bnc    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 499925 INFO  (TEST-StressHdfsTest.test-seed#[F14E7BEDE03608BB]) [n:127.0.0.1:52555_bnc    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-

[...truncated too long message...]

in-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-authn-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-authz-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-changelog-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-collective-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-event-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-exception-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-journal-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-normalization-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-operational-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-referral-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-schema-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-subtree-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-trigger-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-jdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-jdbm1-2.0.0-M2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-kerberos-codec-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-ldif-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-mavibot-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-protocol-kerberos-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-protocol-ldap-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-protocol-shared-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-xdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/api-all-1.0.0-M20.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/bcprov-jdk15-1.45.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/cglib-nodep-2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/commons-collections-3.2.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/commons-math3-3.4.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/easymock-3.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/ehcache-core-2.4.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/hadoop-common-2.6.0-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/hadoop-hdfs-2.6.0-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/hadoop-minikdc-2.6.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jackson-annotations-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jackson-databind-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jcl-over-slf4j-1.7.7.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jersey-core-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jersey-server-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jetty-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jetty-util-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/mina-core-2.0.0-M5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/objenesis-1.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/analysis/icu/lucene-analyzers-icu-5.4.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-analysis-extras/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/contrib/analysis-extras/lib/icu4j-54.1.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-launcher.jar:/x1/jenkins/.ant/lib/ivy-2.3.0.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-antlr.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-regexp.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-resolver.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-net.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jmf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-oro.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-xalan2.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-logging.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jsch.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit4.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-javamail.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jai.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jdepend.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bsf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-swing.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-netrexx.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bcel.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-testutil.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-log4j.jar:/x1/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.7/lib/tools.jar:/x1/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.2.0.jar com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J1-20151114_020227_471.events @/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J1-20151114_020227_471.suites -stdin
   [junit4] ERROR: JVM J1 ended with an exception: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1503)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:130)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:963)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:960)
   [junit4] 	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4] 	at java.lang.Thread.run(Thread.java:745)

BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: At least one slave process threw an exception, first: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.

Total time: 190 minutes 37 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1015 - Still Failing

[Apache Jenkins Server <je...@builds.apache.org>]

Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1015/

1 tests failed.
FAILED:  org.apache.solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR

Error Message:
Captured an uncaught exception in thread: Thread[id=3099, name=coreZkRegister-780-thread-2, state=RUNNABLE, group=TGRP-LeaderInitiatedRecoveryOnShardRestartTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=3099, name=coreZkRegister-780-thread-2, state=RUNNABLE, group=TGRP-LeaderInitiatedRecoveryOnShardRestartTest]
Caused by: java.lang.AssertionError
	at __randomizedtesting.SeedInfo.seed([345BA5D120782BD9]:0)
	at org.apache.solr.cloud.ZkController.updateLeaderInitiatedRecoveryState(ZkController.java:2133)
	at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:434)
	at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)
	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)
	at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:346)
	at org.apache.solr.cloud.ZkController.joinElection(ZkController.java:1113)
	at org.apache.solr.cloud.ZkController.register(ZkController.java:926)
	at org.apache.solr.cloud.ZkController.register(ZkController.java:881)
	at org.apache.solr.core.ZkContainer$2.run(ZkContainer.java:183)
	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 10457 lines...]
   [junit4] Suite: org.apache.solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_345BA5D120782BD9-001/init-core-data-001
   [junit4]   2> 662956 INFO  (SUITE-LeaderInitiatedRecoveryOnShardRestartTest-seed#[345BA5D120782BD9]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   2> 662962 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 662962 INFO  (Thread-625) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 662962 INFO  (Thread-625) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 663062 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.ZkTestServer start zk server on port:49107
   [junit4]   2> 663063 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 663063 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 663066 INFO  (zkCallback-885-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@454c2a6d name:ZooKeeperConnection Watcher:127.0.0.1:49107 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 663067 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 663067 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 663067 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 663071 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 663072 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 663074 INFO  (zkCallback-886-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@19461ed name:ZooKeeperConnection Watcher:127.0.0.1:49107/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 663074 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 663074 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 663074 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 663077 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 663078 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 663080 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 663082 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 663083 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 663088 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 663088 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 663091 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 663091 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 663093 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 663093 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 663095 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 663095 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 663098 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 663098 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 663100 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 663100 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 663103 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 663103 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 663107 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 663108 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 663110 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 663110 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 663111 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 663112 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 663429 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_345BA5D120782BD9-001/control-001/cores/collection1
   [junit4]   2> 663431 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 663434 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@7b5469f5{/,null,AVAILABLE}
   [junit4]   2> 663436 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.e.j.s.ServerConnector Started ServerConnector@e0050{HTTP/1.1}{127.0.0.1:33680}
   [junit4]   2> 663437 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.e.j.s.Server Started @666358ms
   [junit4]   2> 663437 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_345BA5D120782BD9-001/control-001/cores, hostContext=/, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_345BA5D120782BD9-001/tempDir-001/control/data, hostPort=33680}
   [junit4]   2> 663437 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@7b3cb2c6
   [junit4]   2> 663437 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_345BA5D120782BD9-001/control-001/'
   [junit4]   2> 663437 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 663438 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 663456 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 663457 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 663459 INFO  (zkCallback-887-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@65fc9df7 name:ZooKeeperConnection Watcher:127.0.0.1:49107/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 663459 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 663473 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 663475 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 663476 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_345BA5D120782BD9-001/control-001/solr.xml
   [junit4]   2> 663490 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_345BA5D120782BD9-001/control-001/cores
   [junit4]   2> 663490 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.CoreContainer New CoreContainer 1758502352
   [junit4]   2> 663490 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_345BA5D120782BD9-001/control-001/]
   [junit4]   2> 663490 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_345BA5D120782BD9-001/control-001/lib
   [junit4]   2> 663491 WARN  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_345BA5D120782BD9-001/control-001/lib).
   [junit4]   2> 663500 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 663501 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 663501 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 663501 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 663501 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:49107/solr
   [junit4]   2> 663502 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 663502 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 663502 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 663504 INFO  (zkCallback-889-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@cf76937 name:ZooKeeperConnection Watcher:127.0.0.1:49107 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 663504 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 663505 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 663508 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [n:127.0.0.1:33680_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 663511 INFO  (zkCallback-890-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1c029382 name:ZooKeeperConnection Watcher:127.0.0.1:49107/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 663511 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [n:127.0.0.1:33680_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 663513 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [n:127.0.0.1:33680_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 663516 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [n:127.0.0.1:33680_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 663520 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [n:127.0.0.1:33680_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 663525 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [n:127.0.0.1:33680_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 663530 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [n:127.0.0.1:33680_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 663532 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [n:127.0.0.1:33680_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 663534 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [n:127.0.0.1:33680_    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 663535 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [n:127.0.0.1:33680_    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 663537 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [n:127.0.0.1:33680_    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 663541 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [n:127.0.0.1:33680_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:33680_
   [junit4]   2> 663541 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [n:127.0.0.1:33680_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:33680_
   [junit4]   2> 663545 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [n:127.0.0.1:33680_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 663547 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [n:127.0.0.1:33680_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 663549 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [n:127.0.0.1:33680_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 663549 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [n:127.0.0.1:33680_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94854745708953604-127.0.0.1:33680_-n_0000000000
   [junit4]   2> 663551 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [n:127.0.0.1:33680_    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:33680_
   [junit4]   2> 663551 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [n:127.0.0.1:33680_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 663553 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [n:127.0.0.1:33680_    ] o.a.s.c.Overseer Overseer (id=94854745708953604-127.0.0.1:33680_-n_0000000000) starting
   [junit4]   2> 663557 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [n:127.0.0.1:33680_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 663570 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [n:127.0.0.1:33680_    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 663570 INFO  (OverseerCollectionConfigSetProcessor-94854745708953604-127.0.0.1:33680_-n_0000000000) [n:127.0.0.1:33680_    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 663570 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [n:127.0.0.1:33680_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 663571 INFO  (OverseerStateUpdate-94854745708953604-127.0.0.1:33680_-n_0000000000) [n:127.0.0.1:33680_    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 663593 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [n:127.0.0.1:33680_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 663593 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [n:127.0.0.1:33680_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 663594 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [n:127.0.0.1:33680_    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_345BA5D120782BD9-001/control-001/cores
   [junit4]   2> 663597 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [n:127.0.0.1:33680_    ] o.a.s.c.SolrCore Created CoreDescriptor: {loadOnStartup=true, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_345BA5D120782BD9-001/control-001/cores/collection1/, transient=false, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_345BA5D120782BD9-001/control-001/cores/collection1, schema=schema.xml, collection=control_collection, name=collection1, config=solrconfig.xml, coreNodeName=, dataDir=data/, shard=, configSetProperties=configsetprops.json}
   [junit4]   2> 663597 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [n:127.0.0.1:33680_    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_345BA5D120782BD9-001/control-001/cores/collection1/
   [junit4]   2> 663597 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [n:127.0.0.1:33680_    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 663599 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [n:127.0.0.1:33680_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 663599 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [n:127.0.0.1:33680_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 663599 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 663600 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 663602 INFO  (OverseerStateUpdate-94854745708953604-127.0.0.1:33680_-n_0000000000) [n:127.0.0.1:33680_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "state":"down",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "node_name":"127.0.0.1:33680_",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:33680"} current state version: 0
   [junit4]   2> 663603 INFO  (OverseerStateUpdate-94854745708953604-127.0.0.1:33680_-n_0000000000) [n:127.0.0.1:33680_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "state":"down",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "node_name":"127.0.0.1:33680_",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:33680"}
   [junit4]   2> 663603 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 663603 INFO  (OverseerStateUpdate-94854745708953604-127.0.0.1:33680_-n_0000000000) [n:127.0.0.1:33680_    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 663604 INFO  (OverseerStateUpdate-94854745708953604-127.0.0.1:33680_-n_0000000000) [n:127.0.0.1:33680_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 663605 INFO  (zkCallback-890-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 664604 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 664604 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 664604 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 664604 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 664605 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 664605 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_345BA5D120782BD9-001/control-001/cores/collection1/'
   [junit4]   2> 664605 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 664605 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 664622 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 664628 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 664642 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 664671 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 664673 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 664697 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 665013 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 665021 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 665023 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 665033 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 665037 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 665041 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 665044 WARN  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 665044 WARN  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 665044 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 665045 WARN  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 665045 WARN  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 665046 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 665046 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 665046 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 665047 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_345BA5D120782BD9-001/control-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 665047 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@4074ca7c
   [junit4]   2> 665048 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_345BA5D120782BD9-001/control-001/cores/collection1/data
   [junit4]   2> 665048 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_345BA5D120782BD9-001/control-001/cores/collection1/data/index/
   [junit4]   2> 665048 WARN  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_345BA5D120782BD9-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 665048 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_345BA5D120782BD9-001/control-001/cores/collection1/data/index
   [junit4]   2> 665048 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=18, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.3526404574292845]
   [junit4]   2> 665049 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@7d03d0ff lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@58e5578),segFN=segments_1,generation=1}
   [junit4]   2> 665049 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 665049 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_345BA5D120782BD9-001/control-001/cores/collection1/data/
   [junit4]   2> 665050 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_345BA5D120782BD9-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 665057 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 665057 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 665057 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 665057 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 665057 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 665058 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 665058 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 665058 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 665058 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 665058 WARN  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
   [junit4]   2> 665061 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 665063 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 665065 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 665066 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 665071 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: standard,/update/json/docs,/update/csv,/get,/schema,/config,/admin/system,/admin/threads,/admin/ping,/update/json,/admin/plugins,/admin/segments,/admin/properties,/admin/file,/dump,/update,/admin/luke,/admin/mbeans,/admin/logging,/replication
   [junit4]   2> 665073 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 665073 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 665073 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 665074 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 665074 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 665075 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=18, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.34662299133172625]
   [junit4]   2> 665075 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@7d03d0ff lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@58e5578),segFN=segments_1,generation=1}
   [junit4]   2> 665075 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 665075 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@75fa6e9[collection1] main
   [junit4]   2> 665076 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 665077 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 665077 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 665077 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 665077 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 665077 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 665078 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 665078 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 665078 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 665078 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 665080 INFO  (searcherExecutor-743-thread-1-processing-n:127.0.0.1:33680_ r:core_node1 x:collection1 c:control_collection s:shard1) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@75fa6e9[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 665080 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 665080 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 665080 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 665080 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1517675933562044416
   [junit4]   2> 665082 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 2.0ms to seed version buckets with highest version 1517675933562044416
   [junit4]   2> 665082 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 665083 INFO  (coreLoadExecutor-742-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 665083 INFO  (coreZkRegister-736-thread-1-processing-n:127.0.0.1:33680_ r:core_node1 x:collection1 c:control_collection s:shard1) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:33680 collection:control_collection shard:shard1
   [junit4]   2> 665084 INFO  (coreZkRegister-736-thread-1-processing-n:127.0.0.1:33680_ r:core_node1 x:collection1 c:control_collection s:shard1) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 665089 INFO  (coreZkRegister-736-thread-1-processing-n:127.0.0.1:33680_ r:core_node1 x:collection1 c:control_collection s:shard1) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94854745708953604-core_node1-n_0000000000
   [junit4]   2> 665090 INFO  (coreZkRegister-736-thread-1-processing-n:127.0.0.1:33680_ r:core_node1 x:collection1 c:control_collection s:shard1) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 665093 INFO  (coreZkRegister-736-thread-1-processing-n:127.0.0.1:33680_ r:core_node1 x:collection1 c:control_collection s:shard1) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 665093 INFO  (coreZkRegister-736-thread-1-processing-n:127.0.0.1:33680_ r:core_node1 x:collection1 c:control_collection s:shard1) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 665093 INFO  (coreZkRegister-736-thread-1-processing-n:127.0.0.1:33680_ r:core_node1 x:collection1 c:control_collection s:shard1) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:33680/collection1/
   [junit4]   2> 665093 INFO  (coreZkRegister-736-thread-1-processing-n:127.0.0.1:33680_ r:core_node1 x:collection1 c:control_collection s:shard1) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 665093 INFO  (coreZkRegister-736-thread-1-processing-n:127.0.0.1:33680_ r:core_node1 x:collection1 c:control_collection s:shard1) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:33680/collection1/ has no replicas
   [junit4]   2> 665093 INFO  (coreZkRegister-736-thread-1-processing-n:127.0.0.1:33680_ r:core_node1 x:collection1 c:control_collection s:shard1) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:33680/collection1/ shard1
   [junit4]   2> 665094 INFO  (OverseerStateUpdate-94854745708953604-127.0.0.1:33680_-n_0000000000) [n:127.0.0.1:33680_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 665094 INFO  (coreZkRegister-736-thread-1-processing-n:127.0.0.1:33680_ r:core_node1 x:collection1 c:control_collection s:shard1) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 665099 INFO  (coreZkRegister-736-thread-1-processing-n:127.0.0.1:33680_ r:core_node1 x:collection1 c:control_collection s:shard1) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 665101 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 665102 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 665104 INFO  (OverseerStateUpdate-94854745708953604-127.0.0.1:33680_-n_0000000000) [n:127.0.0.1:33680_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:33680",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 1
   [junit4]   2> 665104 INFO  (zkCallback-892-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@59b9bc5 name:ZooKeeperConnection Watcher:127.0.0.1:49107/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 665104 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 665105 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 665105 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 665110 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 665110 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with stateFormat=2
   [junit4]   2> 665110 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 665111 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 665114 INFO  (zkCallback-893-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@37d96a49 name:ZooKeeperConnection Watcher:127.0.0.1:49107/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 665114 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 665114 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 665117 INFO  (OverseerStateUpdate-94854745708953604-127.0.0.1:33680_-n_0000000000) [n:127.0.0.1:33680_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"create",
   [junit4]   2>   "name":"collection1",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "stateFormat":"2"} current state version: 1
   [junit4]   2> 665117 INFO  (OverseerStateUpdate-94854745708953604-127.0.0.1:33680_-n_0000000000) [n:127.0.0.1:33680_    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 665121 INFO  (zkCallback-892-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 665121 INFO  (zkCallback-890-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 665154 INFO  (coreZkRegister-736-thread-1-processing-n:127.0.0.1:33680_ r:core_node1 x:collection1 c:control_collection s:shard1) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:33680/collection1/ and leader is http://127.0.0.1:33680/collection1/
   [junit4]   2> 665154 INFO  (coreZkRegister-736-thread-1-processing-n:127.0.0.1:33680_ r:core_node1 x:collection1 c:control_collection s:shard1) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:33680
   [junit4]   2> 665154 INFO  (coreZkRegister-736-thread-1-processing-n:127.0.0.1:33680_ r:core_node1 x:collection1 c:control_collection s:shard1) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 665154 INFO  (coreZkRegister-736-thread-1-processing-n:127.0.0.1:33680_ r:core_node1 x:collection1 c:control_collection s:shard1) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 665154 INFO  (coreZkRegister-736-thread-1-processing-n:127.0.0.1:33680_ r:core_node1 x:collection1 c:control_collection s:shard1) [n:127.0.0.1:33680_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 665156 INFO  (OverseerStateUpdate-94854745708953604-127.0.0.1:33680_-n_0000000000) [n:127.0.0.1:33680_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "state":"active",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "node_name":"127.0.0.1:33680_",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "base_url":"http://127.0.0.1:33680"} current state version: 2
   [junit4]   2> 665157 INFO  (OverseerStateUpdate-94854745708953604-127.0.0.1:33680_-n_0000000000) [n:127.0.0.1:33680_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "state":"active",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "node_name":"127.0.0.1:33680_",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "base_url":"http://127.0.0.1:33680"}
   [junit4]   2> 665161 INFO  (OverseerStateUpdate-94854745708953604-127.0.0.1:33680_-n_0000000000) [n:127.0.0.1:33680_    ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/collection1/state.json
   [junit4]   2> 665272 INFO  (zkCallback-892-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 665272 INFO  (zkCallback-890-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 665462 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_345BA5D120782BD9-001/shard-1-001/cores/collection1
   [junit4]   2> 665463 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_345BA5D120782BD9-001/shard-1-001
   [junit4]   2> 665464 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 665467 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@567f2d92{/,null,AVAILABLE}
   [junit4]   2> 665467 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.e.j.s.ServerConnector Started ServerConnector@179a499{HTTP/1.1}{127.0.0.1:33857}
   [junit4]   2> 665468 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.e.j.s.Server Started @668389ms
   [junit4]   2> 665468 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solrconfig=solrconfig.xml, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_345BA5D120782BD9-001/tempDir-001/jetty1, hostPort=33857, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_345BA5D120782BD9-001/shard-1-001/cores, hostContext=/}
   [junit4]   2> 665468 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@7b3cb2c6
   [junit4]   2> 665468 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_345BA5D120782BD9-001/shard-1-001/'
   [junit4]   2> 665469 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 665469 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 665490 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 665490 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 665492 INFO  (zkCallback-894-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@794e578 name:ZooKeeperConnection Watcher:127.0.0.1:49107/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 665493 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 665493 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 665495 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 665496 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_345BA5D120782BD9-001/shard-1-001/solr.xml
   [junit4]   2> 665517 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_345BA5D120782BD9-001/shard-1-001/cores
   [junit4]   2> 665518 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.CoreContainer New CoreContainer 812957423
   [junit4]   2> 665518 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_345BA5D120782BD9-001/shard-1-001/]
   [junit4]   2> 665518 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_345BA5D120782BD9-001/shard-1-001/lib
   [junit4]   2> 665518 WARN  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_345BA5D120782BD9-001/shard-1-001/lib).
   [junit4]   2> 665528 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 665530 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 665530 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 665530 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 665530 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:49107/solr
   [junit4]   2> 665530 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 665530 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 665531 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 665533 INFO  (zkCallback-896-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@601fcce7 name:ZooKeeperConnection Watcher:127.0.0.1:49107 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 665534 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 665535 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 665538 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [n:127.0.0.1:33857_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 665541 INFO  (zkCallback-897-thread-1-processing-n:127.0.0.1:33857_) [n:127.0.0.1:33857_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@11102445 name:ZooKeeperConnection Watcher:127.0.0.1:49107/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 665541 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [n:127.0.0.1:33857_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 665549 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [n:127.0.0.1:33857_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 666560 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [n:127.0.0.1:33857_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:33857_
   [junit4]   2> 666560 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [n:127.0.0.1:33857_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:33857_
   [junit4]   2> 666561 INFO  (zkCallback-890-thread-1-processing-n:127.0.0.1:33680_) [n:127.0.0.1:33680_    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 666561 INFO  (zkCallback-892-thread-1) [    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 666562 INFO  (zkCallback-897-thread-1-processing-n:127.0.0.1:33857_) [n:127.0.0.1:33857_    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 666565 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [n:127.0.0.1:33857_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 666568 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [n:127.0.0.1:33857_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94854745708953609-127.0.0.1:33857_-n_0000000001
   [junit4]   2> 666569 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [n:127.0.0.1:33857_    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/94854745708953604-127.0.0.1:33680_-n_0000000000 to know if I could be the leader
   [junit4]   2> 666580 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [n:127.0.0.1:33857_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 666580 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [n:127.0.0.1:33857_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 666584 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [n:127.0.0.1:33857_    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_345BA5D120782BD9-001/shard-1-001/cores
   [junit4]   2> 666587 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [n:127.0.0.1:33857_    ] o.a.s.c.SolrCore Created CoreDescriptor: {name=collection1, shard=, loadOnStartup=true, schema=schema.xml, transient=false, dataDir=data/, configSetProperties=configsetprops.json, coreNodeName=, config=solrconfig.xml, collection=collection1, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_345BA5D120782BD9-001/shard-1-001/cores/collection1/, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_345BA5D120782BD9-001/shard-1-001/cores/collection1}
   [junit4]   2> 666587 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [n:127.0.0.1:33857_    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_345BA5D120782BD9-001/shard-1-001/cores/collection1/
   [junit4]   2> 666587 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [n:127.0.0.1:33857_    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 666589 INFO  (coreLoadExecutor-753-thread-1-processing-n:127.0.0.1:33857_) [n:127.0.0.1:33857_ c:collection1   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 666590 INFO  (coreLoadExecutor-753-thread-1-processing-n:127.0.0.1:33857_) [n:127.0.0.1:33857_ c:collection1   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 666590 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [n:127.0.0.1:33857_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 666590 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [n:127.0.0.1:33857_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 666594 INFO  (coreLoadExecutor-753-thread-1-processing-n:127.0.0.1:33857_) [n:127.0.0.1:33857_ c:collection1   x:collection1] o.a.s.c.ZkController Registering watch for collection collection1
   [junit4]   2> 666594 INFO  (coreLoadExecutor-753-thread-1-processing-n:127.0.0.1:33857_) [n:127.0.0.1:33857_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader addZkWatch collection1
   [junit4]   2> 666594 INFO  (OverseerStateUpdate-94854745708953604-127.0.0.1:33680_-n_0000000000) [n:127.0.0.1:33680_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "state":"down",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "node_name":"127.0.0.1:33857_",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:33857"} current state version: 3
   [junit4]   2> 666595 INFO  (OverseerStateUpdate-94854745708953604-127.0.0.1:33680_-n_0000000000) [n:127.0.0.1:33680_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "state":"down",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "node_name":"127.0.0.1:33857_",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:33857"}
   [junit4]   2> 666595 INFO  (OverseerStateUpdate-94854745708953604-127.0.0.1:33680_-n_0000000000) [n:127.0.0.1:33680_    ] o.a.s.c.o.ReplicaMutator Collection already exists with numShards=1
   [junit4]   2> 666595 INFO  (OverseerStateUpdate-94854745708953604-127.0.0.1:33680_-n_0000000000) [n:127.0.0.1:33680_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 666597 INFO  (coreLoadExecutor-753-thread-1-processing-n:127.0.0.1:33857_) [n:127.0.0.1:33857_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Add data for collection1 ver 0 
   [junit4]   2> 666597 INFO  (coreLoadExecutor-753-thread-1-processing-n:127.0.0.1:33857_) [n:127.0.0.1:33857_ c:collection1   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 666699 INFO  (OverseerStateUpdate-94854745708953604-127.0.0.1:33680_-n_0000000000) [n:127.0.0.1:33680_    ] o.a.s.c.o.ZkStateWriter going to update_collection /collections/collection1/state.json version: 0
   [junit4]   2> 666699 INFO  (zkCallback-897-thread-1-processing-n:127.0.0.1:33857_) [n:127.0.0.1:33857_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json for collection collection1 has occurred - updating... (live nodes size: 2)
   [junit4]   2> 666700 INFO  (zkCallback-897-thread-1-processing-n:127.0.0.1:33857_) [n:127.0.0.1:33857_    ] o.a.s.c.c.ZkStateReader Updating data for collection1 from 0 to 1 
   [junit4]   2> 667597 INFO  (coreLoadExecutor-753-thread-1-processing-n:127.0.0.1:33857_) [n:127.0.0.1:33857_ c:collection1   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 667597 INFO  (coreLoadExecutor-753-thread-1-processing-n:127.0.0.1:33857_) [n:127.0.0.1:33857_ c:collection1   x:collection1] o.a.s.c.ZkController Check for collection zkNode:collection1
   [junit4]   2> 667598 INFO  (coreLoadExecutor-753-thread-1-processing-n:127.0.0.1:33857_) [n:127.0.0.1:33857_ c:collection1   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 667598 INFO  (coreLoadExecutor-753-thread-1-processing-n:127.0.0.1:33857_) [n:127.0.0.1:33857_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/collection1
   [junit4]   2> 667598 INFO  (coreLoadExecutor-753-thread-1-processing-n:127.0.0.1:33857_) [n:127.0.0.1:33857_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 667598 INFO  (coreLoadExecutor-753-thread-1-processing-n:127.0.0.1:33857_) [n:127.0.0.1:33857_ c:collection1   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_345BA5D120782BD9-001/shard-1-001/cores/collection1/'
   [junit4]   2> 667599 INFO  (coreLoadExecutor-753-thread-1-processing-n:127.0.0.1:33857_) [n:127.0.0.1:33857_ c:collection1   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 667599 INFO  (coreLoadExecutor-753-thread-1-processing-n:127.0.0.1:33857_) [n:127.0.0.1:33857_ c:collection1   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 667615 INFO  (coreLoadExecutor-753-thread-1-processing-n:127.0.0.1:33857_) [n:127.0.0.1:33857_ c:collection1   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 667622 INFO  (coreLoadExecutor-753-thread-1-processing-n:127.0.0.1:33857_) [n:127.0.0.1:33857_ c:collection1   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 667636 INFO  (coreLoadExecutor-753-thread-1-processing-n:127.0.0.1:33857_) [n:127.0.0.1:33857_ c:collection1   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 667681 INFO  (coreLoadExecutor-753-thread-1-processing-n:127.0.0.1:33857_) [n:127.0.0.1:33857_ c:coll

[...truncated too long message...]

.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.delete(ZooKeeper.java:873)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$2.execute(SolrZkClient.java:244)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$2.execute(SolrZkClient.java:241)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:61)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.delete(SolrZkClient.java:241)
   [junit4]   2> 	at org.apache.solr.cloud.ElectionContext.cancelElection(ElectionContext.java:85)
   [junit4]   2> 	at org.apache.solr.cloud.ShardLeaderElectionContextBase.cancelElection(ElectionContext.java:166)
   [junit4]   2> 	at org.apache.solr.cloud.ShardLeaderElectionContext.cancelElection(ElectionContext.java:268)
   [junit4]   2> 	at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:450)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.access$200(LeaderElector.java:58)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector$ElectionWatcher.process(LeaderElector.java:388)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$3$1.run(SolrZkClient.java:264)
   [junit4]   2> 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
   [junit4]   2> 	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
   [junit4]   2> 	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 1027516 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.e.j.s.ServerConnector Stopped ServerConnector@626bcfa6{HTTP/1.1}{127.0.0.1:32811}
   [junit4]   2> 1027517 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@221831c6{/,null,UNAVAILABLE}
   [junit4]   2> 1027519 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9]) [n:127.0.0.1:32811_ c:all_in_lir s:shard1 r:core_node2 x:all_in_lir_shard1_replica2] o.a.s.c.ZkTestServer connecting to 127.0.0.1:49107 49107
   [junit4]   2> 1027783 INFO  (Thread-625) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:49107 49107
   [junit4]   2> 1027785 WARN  (Thread-625) [    ] o.a.s.c.ZkTestServer Watch limit violations: 
   [junit4]   2> Maximum concurrent create/delete watches above limit:
   [junit4]   2> 
   [junit4]   2> 	6	/solr/aliases.json
   [junit4]   2> 	5	/solr/configs/conf1
   [junit4]   2> 	5	/solr/security.json
   [junit4]   2> 
   [junit4]   2> Maximum concurrent data watches above limit:
   [junit4]   2> 
   [junit4]   2> 	6	/solr/clusterstate.json
   [junit4]   2> 	4	/solr/collections/collection1/state.json
   [junit4]   2> 	4	/solr/collections/all_in_lir/state.json
   [junit4]   2> 	3	/solr/overseer_elect/election/94854745708953604-127.0.0.1:33680_-n_0000000000
   [junit4]   2> 
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2> 
   [junit4]   2> 	6	/solr/collections
   [junit4]   2> 	4	/solr/live_nodes
   [junit4]   2> 	3	/solr/overseer/collection-queue-work
   [junit4]   2> 	3	/solr/overseer/queue
   [junit4]   2> 
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=LeaderInitiatedRecoveryOnShardRestartTest -Dtests.method=testRestartWithAllInLIR -Dtests.seed=345BA5D120782BD9 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=da -Dtests.timezone=ART -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR    365s J2 | LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR <<<
   [junit4]    > Throwable #1: java.lang.AssertionError: There are still nodes recoverying - waited for 330 seconds
   [junit4]    > 	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:170)
   [junit4]    > 	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:131)
   [junit4]    > 	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:126)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.waitForRecoveriesToFinish(AbstractFullDistribZkTestBase.java:830)
   [junit4]    > 	at org.apache.solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR(LeaderInitiatedRecoveryOnShardRestartTest.java:81)
   [junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:963)
   [junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:938)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)Throwable #2: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=3099, name=coreZkRegister-780-thread-2, state=RUNNABLE, group=TGRP-LeaderInitiatedRecoveryOnShardRestartTest]
   [junit4]    > Caused by: java.lang.AssertionError
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([345BA5D120782BD9]:0)
   [junit4]    > 	at org.apache.solr.cloud.ZkController.updateLeaderInitiatedRecoveryState(ZkController.java:2133)
   [junit4]    > 	at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:434)
   [junit4]    > 	at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)
   [junit4]    > 	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)
   [junit4]    > 	at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:346)
   [junit4]    > 	at org.apache.solr.cloud.ZkController.joinElection(ZkController.java:1113)
   [junit4]    > 	at org.apache.solr.cloud.ZkController.register(ZkController.java:926)
   [junit4]    > 	at org.apache.solr.cloud.ZkController.register(ZkController.java:881)
   [junit4]    > 	at org.apache.solr.core.ZkContainer$2.run(ZkContainer.java:183)
   [junit4]    > 	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)
   [junit4]    > 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   [junit4]    > 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 1027792 INFO  (SUITE-LeaderInitiatedRecoveryOnShardRestartTest-seed#[345BA5D120782BD9]-worker) [n:127.0.0.1:32811_ c:all_in_lir s:shard1 r:core_node2 x:all_in_lir_shard1_replica2] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_345BA5D120782BD9-001
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene54): {}, docValues:{}, sim=RandomSimilarityProvider(queryNorm=false,coord=yes): {}, locale=da, timezone=ART
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_80 (64-bit)/cpus=4,threads=1,free=124801568,total=409468928
   [junit4]   2> NOTE: All tests run in this JVM: [DirectUpdateHandlerOptimizeTest, CoreAdminRequestStatusTest, TestFastWriter, VersionInfoTest, TestObjectReleaseTracker, TestSolrDynamicMBean, SampleTest, TestSolrQueryParserResource, TestCharFilters, TestStressReorder, DocExpirationUpdateProcessorFactoryTest, TestRemoveLastDynamicCopyField, ShardSplitTest, QueryParsingTest, TestRestoreCore, DistributedFacetPivotLongTailTest, TestFileDictionaryLookup, TestCopyFieldCollectionResource, TestDownShardTolerantSearch, ChangedSchemaMergeTest, UpdateParamsTest, TestCloudManagedSchemaConcurrent, ShowFileRequestHandlerTest, LeaderInitiatedRecoveryOnShardRestartTest]
   [junit4] Completed [109/545] on J2 in 364.87s, 1 test, 1 error <<< FAILURES!

[...truncated 1413 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 545 suites (7 ignored), 2158 tests, 1 error, 34 ignored (20 assumptions) [seed: 345BA5D120782BD9]

Total time: 213 minutes 35 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1014 - Still Failing

[Apache Jenkins Server <je...@builds.apache.org>]

Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1014/

1 tests failed.
FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=15513, name=collection4, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=15513, name=collection4, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:35277/pgsc/w: collection already exists: awholynewstresscollection_collection4_1
	at __randomizedtesting.SeedInfo.seed([B2E09DC85F2096EF]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1573)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1594)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:876)




Build Log:
[...truncated 10593 lines...]
   [junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B2E09DC85F2096EF-001/init-core-data-001
   [junit4]   2> 1232913 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[B2E09DC85F2096EF]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (false)
   [junit4]   2> 1232913 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[B2E09DC85F2096EF]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /pgsc/w
   [junit4]   2> 1232918 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 1232918 INFO  (Thread-9565) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1232918 INFO  (Thread-9565) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 1233018 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.ZkTestServer start zk server on port:45614
   [junit4]   2> 1233018 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1233019 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1233022 INFO  (zkCallback-1100-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@339f7f3b name:ZooKeeperConnection Watcher:127.0.0.1:45614 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1233022 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1233022 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1233022 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 1233025 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1233025 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1233026 INFO  (zkCallback-1101-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@42936b3 name:ZooKeeperConnection Watcher:127.0.0.1:45614/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1233026 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1233027 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1233027 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 1233028 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 1233030 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 1233031 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 1233032 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 1233032 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 1233035 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 1233035 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 1233037 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1233038 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1233039 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 1233039 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 1233041 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 1233041 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 1233042 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 1233043 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 1233044 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 1233044 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 1233046 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1233046 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1233048 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1233048 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1233050 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 1233050 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 1233051 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 1233051 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 1233054 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1233054 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1233055 INFO  (zkCallback-1102-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2cdea7bc name:ZooKeeperConnection Watcher:127.0.0.1:45614/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1233055 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1233055 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1233354 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B2E09DC85F2096EF-001/control-001/cores/collection1
   [junit4]   2> 1233355 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 1233357 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@2548151d{/pgsc/w,null,AVAILABLE}
   [junit4]   2> 1233364 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.e.j.s.ServerConnector Started ServerConnector@68c9da9e{SSL-http/1.1}{127.0.0.1:43225}
   [junit4]   2> 1233364 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.e.j.s.Server Started @1236539ms
   [junit4]   2> 1233364 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/pgsc/w, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B2E09DC85F2096EF-001/control-001/cores, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B2E09DC85F2096EF-001/tempDir-001/control/data, hostPort=43225}
   [junit4]   2> 1233364 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@7b3cb2c6
   [junit4]   2> 1233365 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B2E09DC85F2096EF-001/control-001/'
   [junit4]   2> 1233365 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1233365 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1233380 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1233380 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1233381 INFO  (zkCallback-1103-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1cb1461 name:ZooKeeperConnection Watcher:127.0.0.1:45614/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1233381 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1233382 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1233383 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1233383 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B2E09DC85F2096EF-001/control-001/solr.xml
   [junit4]   2> 1233396 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B2E09DC85F2096EF-001/control-001/cores
   [junit4]   2> 1233397 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.CoreContainer New CoreContainer 320795723
   [junit4]   2> 1233397 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B2E09DC85F2096EF-001/control-001/]
   [junit4]   2> 1233397 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B2E09DC85F2096EF-001/control-001/lib
   [junit4]   2> 1233397 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B2E09DC85F2096EF-001/control-001/lib).
   [junit4]   2> 1233405 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1233407 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1233409 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1233409 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1233409 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:45614/solr
   [junit4]   2> 1233409 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 1233409 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1233410 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1233411 INFO  (zkCallback-1105-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@29822494 name:ZooKeeperConnection Watcher:127.0.0.1:45614 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1233411 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1233411 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1233413 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [n:127.0.0.1:43225_pgsc%2Fw    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1233414 INFO  (zkCallback-1106-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@e005dcd name:ZooKeeperConnection Watcher:127.0.0.1:45614/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1233414 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [n:127.0.0.1:43225_pgsc%2Fw    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1233415 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [n:127.0.0.1:43225_pgsc%2Fw    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 1233416 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [n:127.0.0.1:43225_pgsc%2Fw    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 1233419 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [n:127.0.0.1:43225_pgsc%2Fw    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 1233421 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [n:127.0.0.1:43225_pgsc%2Fw    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 1233422 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [n:127.0.0.1:43225_pgsc%2Fw    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 1233425 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [n:127.0.0.1:43225_pgsc%2Fw    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 1233426 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [n:127.0.0.1:43225_pgsc%2Fw    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 1233427 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [n:127.0.0.1:43225_pgsc%2Fw    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 1233428 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [n:127.0.0.1:43225_pgsc%2Fw    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 1233429 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [n:127.0.0.1:43225_pgsc%2Fw    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:43225_pgsc%2Fw
   [junit4]   2> 1233430 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [n:127.0.0.1:43225_pgsc%2Fw    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:43225_pgsc%2Fw
   [junit4]   2> 1233431 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [n:127.0.0.1:43225_pgsc%2Fw    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 1233432 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [n:127.0.0.1:43225_pgsc%2Fw    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 1233433 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [n:127.0.0.1:43225_pgsc%2Fw    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1233434 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [n:127.0.0.1:43225_pgsc%2Fw    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94850738663063557-127.0.0.1:43225_pgsc%2Fw-n_0000000000
   [junit4]   2> 1233434 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [n:127.0.0.1:43225_pgsc%2Fw    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:43225_pgsc%2Fw
   [junit4]   2> 1233434 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [n:127.0.0.1:43225_pgsc%2Fw    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 1233436 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [n:127.0.0.1:43225_pgsc%2Fw    ] o.a.s.c.Overseer Overseer (id=94850738663063557-127.0.0.1:43225_pgsc%2Fw-n_0000000000) starting
   [junit4]   2> 1233437 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [n:127.0.0.1:43225_pgsc%2Fw    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 1233443 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [n:127.0.0.1:43225_pgsc%2Fw    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 1233444 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [n:127.0.0.1:43225_pgsc%2Fw    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1233444 INFO  (OverseerCollectionConfigSetProcessor-94850738663063557-127.0.0.1:43225_pgsc%2Fw-n_0000000000) [n:127.0.0.1:43225_pgsc%2Fw    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 1233444 INFO  (OverseerStateUpdate-94850738663063557-127.0.0.1:43225_pgsc%2Fw-n_0000000000) [n:127.0.0.1:43225_pgsc%2Fw    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 1233457 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [n:127.0.0.1:43225_pgsc%2Fw    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1233457 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [n:127.0.0.1:43225_pgsc%2Fw    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1233458 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [n:127.0.0.1:43225_pgsc%2Fw    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B2E09DC85F2096EF-001/control-001/cores
   [junit4]   2> 1233459 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [n:127.0.0.1:43225_pgsc%2Fw    ] o.a.s.c.SolrCore Created CoreDescriptor: {shard=, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B2E09DC85F2096EF-001/control-001/cores/collection1/, configSetProperties=configsetprops.json, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B2E09DC85F2096EF-001/control-001/cores/collection1, coreNodeName=, schema=schema.xml, transient=false, collection=control_collection, loadOnStartup=true, name=collection1, config=solrconfig.xml, dataDir=data/}
   [junit4]   2> 1233459 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [n:127.0.0.1:43225_pgsc%2Fw    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B2E09DC85F2096EF-001/control-001/cores/collection1/
   [junit4]   2> 1233459 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [n:127.0.0.1:43225_pgsc%2Fw    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 1233460 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [n:127.0.0.1:43225_pgsc%2Fw    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 1233460 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [n:127.0.0.1:43225_pgsc%2Fw    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1233460 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 1233460 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1233462 INFO  (OverseerStateUpdate-94850738663063557-127.0.0.1:43225_pgsc%2Fw-n_0000000000) [n:127.0.0.1:43225_pgsc%2Fw    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "shard":null,
   [junit4]   2>   "roles":null,
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "node_name":"127.0.0.1:43225_pgsc%2Fw",
   [junit4]   2>   "base_url":"https://127.0.0.1:43225/pgsc/w"} current state version: 0
   [junit4]   2> 1233462 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 1233462 INFO  (OverseerStateUpdate-94850738663063557-127.0.0.1:43225_pgsc%2Fw-n_0000000000) [n:127.0.0.1:43225_pgsc%2Fw    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "shard":null,
   [junit4]   2>   "roles":null,
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "node_name":"127.0.0.1:43225_pgsc%2Fw",
   [junit4]   2>   "base_url":"https://127.0.0.1:43225/pgsc/w"}
   [junit4]   2> 1233463 INFO  (OverseerStateUpdate-94850738663063557-127.0.0.1:43225_pgsc%2Fw-n_0000000000) [n:127.0.0.1:43225_pgsc%2Fw    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 1233463 INFO  (OverseerStateUpdate-94850738663063557-127.0.0.1:43225_pgsc%2Fw-n_0000000000) [n:127.0.0.1:43225_pgsc%2Fw    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 1233464 INFO  (zkCallback-1106-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1234462 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1234463 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 1234463 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 1234463 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 1234464 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1234464 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B2E09DC85F2096EF-001/control-001/cores/collection1/'
   [junit4]   2> 1234464 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1234464 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1234477 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 1234483 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 1234496 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 1234522 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1234523 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 1234529 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1234752 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 1234759 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 1234761 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 1234770 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1234772 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1234775 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1234775 WARN  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 1234775 WARN  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 1234775 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1234776 WARN  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 1234776 WARN  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 1234777 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 1234777 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 1234777 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 1234777 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B2E09DC85F2096EF-001/control-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 1234778 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@3e52e33f
   [junit4]   2> 1234778 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B2E09DC85F2096EF-001/control-001/cores/collection1/data
   [junit4]   2> 1234778 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B2E09DC85F2096EF-001/control-001/cores/collection1/data/index/
   [junit4]   2> 1234778 WARN  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B2E09DC85F2096EF-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 1234779 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B2E09DC85F2096EF-001/control-001/cores/collection1/data/index
   [junit4]   2> 1234779 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=50, maxMergeAtOnceExplicit=29, maxMergedSegmentMB=73.998046875, floorSegmentMB=1.0302734375, forceMergeDeletesPctAllowed=1.1723843984137305, segmentsPerTier=34.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.605175267747365
   [junit4]   2> 1234781 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(NRTCachingDirectory(MMapDirectory@/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B2E09DC85F2096EF-001/index-MMapDirectory-001 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7ac22fc4; maxCacheMB=0.1884765625 maxMergeSizeMB=0.173828125)),segFN=segments_1,generation=1}
   [junit4]   2> 1234781 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 1234781 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B2E09DC85F2096EF-001/control-001/cores/collection1/data/
   [junit4]   2> 1234782 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B2E09DC85F2096EF-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 1234785 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 1234785 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 1234786 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 1234786 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1234786 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1234786 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 1234786 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1234786 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1234786 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 1234786 WARN  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
   [junit4]   2> 1234789 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1234790 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1234791 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1234792 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1234796 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/segments,/admin/threads,/config,/admin/mbeans,/admin/ping,/admin/logging,/update/csv,/admin/system,/admin/file,/admin/plugins,/dump,/replication,/schema,/update/json,standard,/admin/properties,/get,/admin/luke,/update,/update/json/docs
   [junit4]   2> 1234797 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 1234798 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1234798 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1234799 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 1234799 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 1234799 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=1855868822, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
   [junit4]   2> 1234800 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(NRTCachingDirectory(MMapDirectory@/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B2E09DC85F2096EF-001/index-MMapDirectory-001 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7ac22fc4; maxCacheMB=0.1884765625 maxMergeSizeMB=0.173828125)),segFN=segments_1,generation=1}
   [junit4]   2> 1234800 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 1234801 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@322c6b38[collection1] main
   [junit4]   2> 1234801 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 1234802 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1234802 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 1234802 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 1234802 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 1234802 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1234803 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 1234803 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1234803 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 1234803 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1234803 INFO  (searcherExecutor-3326-thread-1-processing-s:shard1 c:control_collection x:collection1 r:core_node1 n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@322c6b38[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1234803 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 1234804 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 1234804 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 1234804 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1517611820585582592
   [junit4]   2> 1234806 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 2.0ms to seed version buckets with highest version 1517611820585582592
   [junit4]   2> 1234806 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 1234806 INFO  (coreLoadExecutor-3325-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 1234807 INFO  (coreZkRegister-3319-thread-1-processing-s:shard1 c:control_collection x:collection1 r:core_node1 n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:https://127.0.0.1:43225/pgsc/w collection:control_collection shard:shard1
   [junit4]   2> 1234808 INFO  (coreZkRegister-3319-thread-1-processing-s:shard1 c:control_collection x:collection1 r:core_node1 n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 1234811 INFO  (coreZkRegister-3319-thread-1-processing-s:shard1 c:control_collection x:collection1 r:core_node1 n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94850738663063557-core_node1-n_0000000000
   [junit4]   2> 1234811 INFO  (coreZkRegister-3319-thread-1-processing-s:shard1 c:control_collection x:collection1 r:core_node1 n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 1234813 INFO  (coreZkRegister-3319-thread-1-processing-s:shard1 c:control_collection x:collection1 r:core_node1 n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1234813 INFO  (coreZkRegister-3319-thread-1-processing-s:shard1 c:control_collection x:collection1 r:core_node1 n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1234813 INFO  (coreZkRegister-3319-thread-1-processing-s:shard1 c:control_collection x:collection1 r:core_node1 n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:43225/pgsc/w/collection1/
   [junit4]   2> 1234813 INFO  (coreZkRegister-3319-thread-1-processing-s:shard1 c:control_collection x:collection1 r:core_node1 n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 1234813 INFO  (coreZkRegister-3319-thread-1-processing-s:shard1 c:control_collection x:collection1 r:core_node1 n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy https://127.0.0.1:43225/pgsc/w/collection1/ has no replicas
   [junit4]   2> 1234814 INFO  (coreZkRegister-3319-thread-1-processing-s:shard1 c:control_collection x:collection1 r:core_node1 n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:43225/pgsc/w/collection1/ shard1
   [junit4]   2> 1234814 INFO  (OverseerStateUpdate-94850738663063557-127.0.0.1:43225_pgsc%2Fw-n_0000000000) [n:127.0.0.1:43225_pgsc%2Fw    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 1234814 INFO  (coreZkRegister-3319-thread-1-processing-s:shard1 c:control_collection x:collection1 r:core_node1 n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 1234817 INFO  (coreZkRegister-3319-thread-1-processing-s:shard1 c:control_collection x:collection1 r:core_node1 n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 1234819 INFO  (OverseerStateUpdate-94850738663063557-127.0.0.1:43225_pgsc%2Fw-n_0000000000) [n:127.0.0.1:43225_pgsc%2Fw    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"https://127.0.0.1:43225/pgsc/w",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 1
   [junit4]   2> 1234922 INFO  (zkCallback-1106-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1234964 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1234964 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1234966 INFO  (zkCallback-1108-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2226a943 name:ZooKeeperConnection Watcher:127.0.0.1:45614/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1234966 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1234966 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1234966 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1234968 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 1234968 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with stateFormat=2
   [junit4]   2> 1234968 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1234969 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1234969 INFO  (coreZkRegister-3319-thread-1-processing-s:shard1 c:control_collection x:collection1 r:core_node1 n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are https://127.0.0.1:43225/pgsc/w/collection1/ and leader is https://127.0.0.1:43225/pgsc/w/collection1/
   [junit4]   2> 1234970 INFO  (coreZkRegister-3319-thread-1-processing-s:shard1 c:control_collection x:collection1 r:core_node1 n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:43225/pgsc/w
   [junit4]   2> 1234970 INFO  (coreZkRegister-3319-thread-1-processing-s:shard1 c:control_collection x:collection1 r:core_node1 n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 1234970 INFO  (coreZkRegister-3319-thread-1-processing-s:shard1 c:control_collection x:collection1 r:core_node1 n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 1234970 INFO  (coreZkRegister-3319-thread-1-processing-s:shard1 c:control_collection x:collection1 r:core_node1 n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1234970 INFO  (zkCallback-1109-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@67ccc1b9 name:ZooKeeperConnection Watcher:127.0.0.1:45614/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1234971 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1234971 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1234972 INFO  (OverseerStateUpdate-94850738663063557-127.0.0.1:43225_pgsc%2Fw-n_0000000000) [n:127.0.0.1:43225_pgsc%2Fw    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "node_name":"127.0.0.1:43225_pgsc%2Fw",
   [junit4]   2>   "base_url":"https://127.0.0.1:43225/pgsc/w"} current state version: 2
   [junit4]   2> 1234973 INFO  (OverseerStateUpdate-94850738663063557-127.0.0.1:43225_pgsc%2Fw-n_0000000000) [n:127.0.0.1:43225_pgsc%2Fw    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "node_name":"127.0.0.1:43225_pgsc%2Fw",
   [junit4]   2>   "base_url":"https://127.0.0.1:43225/pgsc/w"}
   [junit4]   2> 1234977 INFO  (OverseerStateUpdate-94850738663063557-127.0.0.1:43225_pgsc%2Fw-n_0000000000) [n:127.0.0.1:43225_pgsc%2Fw    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"create",
   [junit4]   2>   "name":"collection1",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "stateFormat":"2"} current state version: 2
   [junit4]   2> 1234977 INFO  (OverseerStateUpdate-94850738663063557-127.0.0.1:43225_pgsc%2Fw-n_0000000000) [n:127.0.0.1:43225_pgsc%2Fw    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 1234977 INFO  (zkCallback-1106-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1234980 INFO  (zkCallback-1108-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1235087 INFO  (OverseerStateUpdate-94850738663063557-127.0.0.1:43225_pgsc%2Fw-n_0000000000) [n:127.0.0.1:43225_pgsc%2Fw    ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/collection1/state.json
   [junit4]   2> 1235274 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B2E09DC85F2096EF-001/shard-1-001/cores/collection1
   [junit4]   2> 1235275 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B2E09DC85F2096EF-001/shard-1-001
   [junit4]   2> 1235276 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 1235277 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@7c393a8f{/pgsc/w,null,AVAILABLE}
   [junit4]   2> 1235283 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.e.j.s.ServerConnector Started ServerConnector@5d0596d2{SSL-http/1.1}{127.0.0.1:35277}
   [junit4]   2> 1235283 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.e.j.s.Server Started @1238458ms
   [junit4]   2> 1235283 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostPort=35277, hostContext=/pgsc/w, solrconfig=solrconfig.xml, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B2E09DC85F2096EF-001/tempDir-001/jetty1, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B2E09DC85F2096EF-001/shard-1-001/cores}
   [junit4]   2> 1235283 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@7b3cb2c6
   [junit4]   2> 1235283 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B2E09DC85F2096EF-001/shard-1-001/'
   [junit4]   2> 1235284 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1235284 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1235299 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1235300 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1235301 INFO  (zkCallback-1110-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7b1405e3 name:ZooKeeperConnection Watcher:127.0.0.1:45614/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1235301 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1235301 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1235302 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1235302 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B2E09DC85F2096EF-001/shard-1-001/solr.xml
   [junit4]   2> 1235316 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B2E09DC85F2096EF-001/shard-1-001/cores
   [junit4]   2> 1235316 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.CoreContainer New CoreContainer 1058685286
   [junit4]   2> 1235316 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B2E09DC85F2096EF-001/shard-1-001/]
   [junit4]   2> 1235316 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B2E09DC85F2096EF-001/shard-1-001/lib
   [junit4]   2> 1235317 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B2E09DC85F2096EF-001/shard-1-001/lib).
   [junit4]   2> 1235325 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1235327 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1235328 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1235328 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1235329 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:45614/solr
   [junit4]   2> 1235329 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 1235329 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1235329 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1235331 INFO  (zkCallback-1112-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@fa76653 name:ZooKeeperConnection Watcher:127.0.0.1:45614 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1235331 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1235331 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1235332 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [n:127.0.0.1:35277_pgsc%2Fw    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1235333 INFO  (zkCallback-1113-thread-1-processing-n:127.0.0.1:35277_pgsc%2Fw) [n:127.0.0.1:35277_pgsc%2Fw    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6be6ce05 name:ZooKeeperConnection Watcher:127.0.0.1:45614/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1235333 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [n:127.0.0.1:35277_pgsc%2Fw    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1235339 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [n:127.0.0.1:35277_pgsc%2Fw    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1236345 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [n:127.0.0.1:35277_pgsc%2Fw    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:35277_pgsc%2Fw
   [junit4]   2> 1236345 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [n:127.0.0.1:35277_pgsc%2Fw    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:35277_pgsc%2Fw
   [junit4]   2> 1236346 INFO  (zkCallback-1108-thread-1) [    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1236346 INFO  (zkCallback-1113-thread-1-processing-n:127.0.0.1:35277_pgsc%2Fw) [n:127.0.0.1:35277_pgsc%2Fw    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1236346 INFO  (zkCallback-1106-thread-1-processing-n:127.0.0.1:43225_pgsc%2Fw) [n:127.0.0.1:43225_pgsc%2Fw    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1236348 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [n:127.0.0.1:35277_pgsc%2Fw    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1236348 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [n:127.0.0.1:35277_pgsc%2Fw    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94850738663063562-127.0.0.1:35277_pgsc%2Fw-n_0000000001
   [junit4]   2> 1236349 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [n:127.0.0.1:35277_pgsc%2Fw    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/94850738663063557-127.0.0.1:43225_pgsc%2Fw-n_0000000000 to know if I could be the leader
   [junit4]   2> 1236363 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [n:127.0.0.1:35277_pgsc%2Fw    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1236363 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [n:127.0.0.1:35277_pgsc%2Fw    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1236364 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [n:127.0.0.1:35277_pgsc%2Fw    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B2E09DC85F2096EF-001/shard-1-001/cores
   [junit4]   2> 1236365 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [n:127.0.0.1:35277_pgsc%2Fw    ] o.a.s.c.SolrCore Created CoreDescriptor: {shard=, schema=schema.xml, collection=collection1, transient=false, loadOnStartup=true, dataDir=data/, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B2E09DC85F2096EF-001/shard-1-001/cores/collection1/, coreNodeName=, configSetProperties=configsetprops.json, name=collection1, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B2E09DC85F2096EF-001/shard-1-001/cores/collection1, config=solrconfig.xml}
   [junit4]   2> 1236365 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [n:127.0.0.1:35277_pgsc%2Fw    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B2E09DC85F2096EF-001/shard-1-001/cores/collection1/
   [junit4]   2> 1236365 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [n:127.0.0.1:35277_pgsc%2Fw    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 1236366 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [n:127.0.0.1:35277_pgsc%2Fw    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 1236366 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B2E09DC85F2096EF]) [n:127.0.0.1:35277_pgsc%2Fw    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1236367 INFO  (coreLoadExecutor-3336-thread-1-processing-n:127.0.0.1:35277_pgsc%2Fw) [n:127.0.0.1:35277_pgsc%2Fw c:collection1   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 1236367 INFO  (coreLoadExecutor-3336-thread-1-processing-n:127.0.0.1:35277_pgsc%2Fw) [n:127.0.0.1:35277_pgsc%2Fw c:collection1   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1236368 INFO  (coreLoadExecutor-3336-thread-1-processing-n:127.0.0.1:35277_pgsc%2Fw) [n:127.0.0.1:35277_pgsc%2Fw c:collection1   x:collection1] o.a.s.c.ZkController Registering watch for collection collection1
   [junit4]   2> 1236368 INFO  (coreLoadExecutor-3336-thread-1-processing-n:127.0.0.1:35277_pgsc%2Fw) [n:127.0.0.1:35277_pgsc%2Fw c:collection1   x:collection1] o.a.s.c.c.ZkStateReader addZkWatch collection1
   [junit4]   2> 1236369 INFO  (OverseerStateUpdate-94850738663063557-127.0.0.1:43225_pgsc%2Fw-n_0000000000) [n:127.0.0.1:43225_pgsc%2Fw    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "shard":null,
   [junit4]   2>   "roles":null,
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "node_name":"127.0.0.1:35277_pgsc%2Fw",
   [junit4]   2>   "base_url":"https://127.0.0.1:35277/pgsc/w"} current state version: 3
   [junit4]   2> 1236369 INFO  (coreLoadExecutor-3336-thread-1-processing-n:127.0.0.1:35277_pgsc%2Fw) [n:127.0.0.1:35277_pgsc%2Fw c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Add data for collection1 ver 0 
   [junit4]   2> 1236369 INFO  (coreLoadExecutor-3336-thread-1-processing-n:127.0.0.1:35277_pgsc%2Fw) [n:127.0.0.1:35277_pgsc%2Fw c:collection1   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 1236369 INFO  (OverseerStateUpdate-94850738663063557-127.0.0.1:43225_pgsc%2Fw-n_0000000000) [n:127.0.0.1:43225_pgsc%2Fw    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "shard":null,
   [junit4]   2>   "roles":null,
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "node_name":"127.0.0.1:35277_pgsc%2Fw",
   [junit4]   2>   "base_url":"https://127.0.0.1:35277/pgsc/w"}
   [junit4]   2> 1236369 INFO  (OverseerStateUpdate-94850738663063557-127.0.0.1:43225_pgsc%2Fw-n_0000000000) [n:127.0.0.1:43225_pgsc%2Fw    ] o.a.s.c.o.ReplicaMutator Collection already exists with numShards=2
   [junit4]   2> 1236370 INFO  (OverseerStateUpdate-94850738663063557-127.0.0.1:43225_pgsc%2Fw-n_0000000000) [n:127.0.0.1:43225_pgsc%2Fw    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard2
   [junit4]   2> 1236472 INFO  (OverseerStateUpdate-94850738663063557-127.0.0.1:43225_pgsc%2Fw-n_0000000000) [n:127.0.0.1:43225_pgsc%2Fw    ] o.a.s.c.o.ZkStateWriter going to update_collection /collections/collection1/state.json version: 0
   [junit4]   2> 1236473 INFO  (zkCallback-1113-thread-1-processing-n:127.0.0.1:35277_pgsc%2Fw) [n:127.0.0.1:35277_pgsc%2Fw    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json for collection collection1 has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1236473 INFO  (zkCallback-1113-thread-1-processing-n:127.0.0.1:35277_pgsc%2Fw) [n:127.0.0.1:35277_pgsc%2Fw    ] o.a.s.c.c.ZkStateReader Updating data for collection1 from 0 to 1 
   [junit4]   2> 1237369 INFO  (coreLoadExecutor-3336-thread-1-processing-n:127.0.0.1:35277_pgsc%2Fw) [n:127.0.0.1:35277_pgsc%2Fw c:collection1   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1237369 INFO  (coreLoadExecutor-3336-thread-1-processing-n:127.0.0.1:35277_pgsc%2Fw) [n:127.0.0.1:35277_pgsc%2Fw c:collection1   x:collection1] o.a.s.c.ZkController Check for collection zkNode:collection1
   [junit4]   2> 1237370 INFO  (coreLoadExecutor-3336-thread-1-processing-n:127.0.0.1:35277_pgsc%2Fw) [n:127.0.0.1:35277_pgsc%2Fw c:collection1   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 1237370 INFO  (coreLoadExecutor-3336-thread-1-processing-n:127.0.0.1:35277_pgsc%2Fw) [n:127.0.0.1:35277_pgsc%2Fw c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/collection1
   [junit4]   2> 1237370 INFO  (coreLoadExecutor-3336-thread-1-processing-n:127.0.0.1:35277_pgsc%2Fw) [n:127.0.0.1:35277_pgsc%2Fw c:collection1   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1237370 INFO  (coreLoadExecutor-3336-thread-1-processing-n:127.0.0.1:35277_pgsc%2Fw) [n:127.0.0.1:35277_pgsc%2Fw c:collection1   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/tes

[...truncated too long message...]

aused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:48443/pgsc/w: collection already exists: awholynewstresscollection_collection2_2
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([B2E09DC85F2096EF]:0)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1573)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1594)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:876)Throwable #5: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=15512, name=collection3, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    > Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:48443/pgsc/w: Could not find collection : awholynewstresscollection_collection3_1
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([B2E09DC85F2096EF]:0)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:883)Throwable #6: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=15510, name=collection1, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    > Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:48443/pgsc/w: collection already exists: awholynewstresscollection_collection1_2
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([B2E09DC85F2096EF]:0)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1573)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1594)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:876)
   [junit4]   2> 1790963 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[B2E09DC85F2096EF]-worker) [n:127.0.0.1:47537_pgsc%2Fw c:addReplicaColl s:shard1 r:core_node7 x:propertyDotName] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene54): {range_facet_i_dv=PostingsFormat(name=LuceneVarGapFixedInterval), multiDefault=PostingsFormat(name=LuceneVarGapFixedInterval), _version_=FST50, intDefault=FST50, a_i1=PostingsFormat(name=LuceneVarGapFixedInterval), timestamp=PostingsFormat(name=LuceneVarGapFixedInterval), range_facet_l=PostingsFormat(name=LuceneVarGapFixedInterval), other_tl1=FST50, id=PostingsFormat(name=Direct), a_t=PostingsFormat(name=LuceneVarGapFixedInterval), text=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), range_facet_l_dv=PostingsFormat(name=Direct)}, docValues:{range_facet_i_dv=DocValuesFormat(name=Direct), timestamp=DocValuesFormat(name=Direct), range_facet_l_dv=DocValuesFormat(name=Lucene54)}, sim=DefaultSimilarity, locale=ar_DZ, timezone=Brazil/East
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_80 (64-bit)/cpus=4,threads=1,free=111490048,total=510656512
   [junit4]   2> NOTE: All tests run in this JVM: [TestStressVersions, TestWordDelimiterFilterFactory, TestPivotHelperCode, TestPartialUpdateDeduplication, AtomicUpdatesTest, TestDistributedMissingSort, DistributedFacetPivotSmallTest, TestMacros, SaslZkACLProviderTest, DistributedTermsComponentTest, SpellCheckComponentTest, TestSchemaResource, ZkSolrClientTest, HLLUtilTest, TestQuerySenderListener, FastVectorHighlighterTest, IndexBasedSpellCheckerTest, TestRawTransformer, SchemaVersionSpecificBehaviorTest, TestExpandComponent, TestMaxScoreQueryParser, ShardRoutingCustomTest, TestSearcherReuse, TestSolr4Spatial2, HighlighterTest, SpellingQueryConverterTest, SpatialHeatmapFacetsTest, TestFastLRUCache, TestTrieFacet, TestTolerantSearch, NotRequiredUniqueKeyTest, PKIAuthenticationIntegrationTest, TermsComponentTest, SpellCheckCollatorTest, TestClassNameShortening, SolrIndexConfigTest, LoggingHandlerTest, DataDrivenBlockJoinTest, TestRecoveryHdfs, StressHdfsTest, DistributedFacetPivotWhiteBoxTest, ClusterStateUpdateTest, OutOfBoxZkACLAndCredentialsProvidersTest, TestClassicSimilarityFactory, DocValuesTest, TestFreeTextSuggestions, TestQuerySenderNoQuery, TestMiniSolrCloudCluster, TermVectorComponentTest, TestBM25SimilarityFactory, TestDistributedGrouping, TestQueryTypes, ResponseHeaderTest, ChaosMonkeyNothingIsSafeTest, TestTrie, SoftAutoCommitTest, TestUpdate, DistributedQueueTest, OverriddenZkACLAndCredentialsProvidersTest, CSVRequestHandlerTest, TestSerializedLuceneMatchVersion, AnalysisErrorHandlingTest, TestCollapseQParserPlugin, TestConfigSetProperties, MultiTermTest, TestSchemaSimilarityResource, TestAuthenticationFramework, ShardRoutingTest, NumericFieldsTest, TestNonDefinedSimilarityFactory, TestRandomMergePolicy, TestCursorMarkWithoutUniqueKey, DirectSolrSpellCheckerTest, AsyncMigrateRouteKeyTest, SolrTestCaseJ4Test, CoreMergeIndexesAdminHandlerTest, ExitableDirectoryReaderTest, TestCustomSort, SimplePostToolTest, TestClusterStateMutator, TestSystemIdResolver, CollectionStateFormat2Test, ZkStateReaderTest, TestMiniSolrCloudClusterBase, DistributedMLTComponentTest, TestUniqueKeyFieldResource, TestInitQParser, TestManagedResource, SortByFunctionTest, ChaosMonkeySafeLeaderTest, BasicDistributedZk2Test, CollectionsAPIDistributedZkTest]
   [junit4] Completed [194/545] on J1 in 558.80s, 1 test, 1 error <<< FAILURES!

[...truncated 1135 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 545 suites (7 ignored), 2158 tests, 1 error, 34 ignored (20 assumptions) [seed: B2E09DC85F2096EF]

Total time: 221 minutes 46 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1013 - Still Failing

[Apache Jenkins Server <je...@builds.apache.org>]

Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1013/

1 tests failed.
FAILED:  org.apache.lucene.analysis.core.TestAllAnalyzersHaveFactories.test

Error Message:
A SPI class of type org.apache.lucene.analysis.util.TokenFilterFactory with name 'SerbianNormalizationRegular' does not exist. You need to add the corresponding JAR file supporting this SPI to your classpath. The current classpath supports the following names: [apostrophe, arabicnormalization, arabicstem, bulgarianstem, brazilianstem, cjkbigram, cjkwidth, soraninormalization, soranistem, commongrams, commongramsquery, dictionarycompoundword, hyphenationcompoundword, decimaldigit, lowercase, stop, type, uppercase, czechstem, germanlightstem, germanminimalstem, germannormalization, germanstem, greeklowercase, greekstem, englishminimalstem, englishpossessive, kstem, porterstem, spanishlightstem, persiannormalization, finnishlightstem, frenchlightstem, frenchminimalstem, irishlowercase, galicianminimalstem, galicianstem, hindinormalization, hindistem, hungarianlightstem, hunspellstem, indonesianstem, indicnormalization, italianlightstem, latvianstem, asciifolding, capitalization, codepointcount, fingerprint, hyphenatedwords, keepword, keywordmarker, keywordrepeat, length, limittokencount, limittokenoffset, limittokenposition, removeduplicates, stemmeroverride, trim, truncate, worddelimiter, scandinavianfolding, scandinaviannormalization, edgengram, ngram, norwegianlightstem, norwegianminimalstem, patternreplace, patterncapturegroup, delimitedpayload, numericpayload, tokenoffsetpayload, typeaspayload, portugueselightstem, portugueseminimalstem, portuguesestem, reversestring, russianlightstem, shingle, snowballporter, serbiannormalization, classic, standard, swedishlightstem, synonym, thaiword, turkishlowercase, elision]

Stack Trace:
java.lang.IllegalArgumentException: A SPI class of type org.apache.lucene.analysis.util.TokenFilterFactory with name 'SerbianNormalizationRegular' does not exist. You need to add the corresponding JAR file supporting this SPI to your classpath. The current classpath supports the following names: [apostrophe, arabicnormalization, arabicstem, bulgarianstem, brazilianstem, cjkbigram, cjkwidth, soraninormalization, soranistem, commongrams, commongramsquery, dictionarycompoundword, hyphenationcompoundword, decimaldigit, lowercase, stop, type, uppercase, czechstem, germanlightstem, germanminimalstem, germannormalization, germanstem, greeklowercase, greekstem, englishminimalstem, englishpossessive, kstem, porterstem, spanishlightstem, persiannormalization, finnishlightstem, frenchlightstem, frenchminimalstem, irishlowercase, galicianminimalstem, galicianstem, hindinormalization, hindistem, hungarianlightstem, hunspellstem, indonesianstem, indicnormalization, italianlightstem, latvianstem, asciifolding, capitalization, codepointcount, fingerprint, hyphenatedwords, keepword, keywordmarker, keywordrepeat, length, limittokencount, limittokenoffset, limittokenposition, removeduplicates, stemmeroverride, trim, truncate, worddelimiter, scandinavianfolding, scandinaviannormalization, edgengram, ngram, norwegianlightstem, norwegianminimalstem, patternreplace, patterncapturegroup, delimitedpayload, numericpayload, tokenoffsetpayload, typeaspayload, portugueselightstem, portugueseminimalstem, portuguesestem, reversestring, russianlightstem, shingle, snowballporter, serbiannormalization, classic, standard, swedishlightstem, synonym, thaiword, turkishlowercase, elision]
	at __randomizedtesting.SeedInfo.seed([150596469108C85C:9D51A99C3FF4A5A4]:0)
	at org.apache.lucene.analysis.util.AnalysisSPILoader.lookupClass(AnalysisSPILoader.java:122)
	at org.apache.lucene.analysis.util.TokenFilterFactory.lookupClass(TokenFilterFactory.java:42)
	at org.apache.lucene.analysis.core.TestAllAnalyzersHaveFactories.test(TestAllAnalyzersHaveFactories.java:153)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1660)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:866)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:902)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:916)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:875)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:777)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:811)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:822)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 2814 lines...]
   [junit4] Suite: org.apache.lucene.analysis.core.TestAllAnalyzersHaveFactories
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestAllAnalyzersHaveFactories -Dtests.method=test -Dtests.seed=150596469108C85C -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=en_ZA -Dtests.timezone=America/Argentina/Mendoza -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR   0.88s J0 | TestAllAnalyzersHaveFactories.test <<<
   [junit4]    > Throwable #1: java.lang.IllegalArgumentException: A SPI class of type org.apache.lucene.analysis.util.TokenFilterFactory with name 'SerbianNormalizationRegular' does not exist. You need to add the corresponding JAR file supporting this SPI to your classpath. The current classpath supports the following names: [apostrophe, arabicnormalization, arabicstem, bulgarianstem, brazilianstem, cjkbigram, cjkwidth, soraninormalization, soranistem, commongrams, commongramsquery, dictionarycompoundword, hyphenationcompoundword, decimaldigit, lowercase, stop, type, uppercase, czechstem, germanlightstem, germanminimalstem, germannormalization, germanstem, greeklowercase, greekstem, englishminimalstem, englishpossessive, kstem, porterstem, spanishlightstem, persiannormalization, finnishlightstem, frenchlightstem, frenchminimalstem, irishlowercase, galicianminimalstem, galicianstem, hindinormalization, hindistem, hungarianlightstem, hunspellstem, indonesianstem, indicnormalization, italianlightstem, latvianstem, asciifolding, capitalization, codepointcount, fingerprint, hyphenatedwords, keepword, keywordmarker, keywordrepeat, length, limittokencount, limittokenoffset, limittokenposition, removeduplicates, stemmeroverride, trim, truncate, worddelimiter, scandinavianfolding, scandinaviannormalization, edgengram, ngram, norwegianlightstem, norwegianminimalstem, patternreplace, patterncapturegroup, delimitedpayload, numericpayload, tokenoffsetpayload, typeaspayload, portugueselightstem, portugueseminimalstem, portuguesestem, reversestring, russianlightstem, shingle, snowballporter, serbiannormalization, classic, standard, swedishlightstem, synonym, thaiword, turkishlowercase, elision]
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([150596469108C85C:9D51A99C3FF4A5A4]:0)
   [junit4]    > 	at org.apache.lucene.analysis.util.AnalysisSPILoader.lookupClass(AnalysisSPILoader.java:122)
   [junit4]    > 	at org.apache.lucene.analysis.util.TokenFilterFactory.lookupClass(TokenFilterFactory.java:42)
   [junit4]    > 	at org.apache.lucene.analysis.core.TestAllAnalyzersHaveFactories.test(TestAllAnalyzersHaveFactories.java:153)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> NOTE: test params are: codec=Lucene54, sim=DefaultSimilarity, locale=en_ZA, timezone=America/Argentina/Mendoza
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_80 (64-bit)/cpus=4,threads=1,free=127645408,total=272629760
   [junit4]   2> NOTE: All tests run in this JVM: [TestBrazilianStemFilterFactory, TestGreekLowerCaseFilterFactory, TestPortugueseMinimalStemFilterFactory, ShingleFilterTest, TestScandinavianNormalizationFilter, DateRecognizerSinkTokenizerTest, TestKeepWordFilter, TestHomonyms, NGramTokenFilterTest, TestItalianLightStemFilter, TestWordlistLoader, TestSoraniStemFilterFactory, TestMorph, TestArabicNormalizationFilter, TestHungarianLightStemFilterFactory, TestPortugueseLightStemFilterFactory, TestNorwegianLightStemFilterFactory, TestSingleTokenTokenFilter, TestGalicianMinimalStemFilter, TestHindiAnalyzer, TestTurkishAnalyzer, TestTruncateTokenFilter, TestFrenchAnalyzer, TestGalicianMinimalStemFilterFactory, TestCzechStemmer, TestPersianNormalizationFilter, TestPortugueseStemFilterFactory, TestPortugueseLightStemFilter, TestWikipediaTokenizerFactory, TestPatternTokenizer, EdgeNGramTokenizerTest, TestMappingCharFilterFactory, TestOptionalCondition, TestSerbianNormalizationFilterFactory, NumericPayloadTokenFilterTest, TestKeywordMarkerFilter, TestHindiFilters, TestBulgarianAnalyzer, TestScandinavianFoldingFilter, TestEnglishAnalyzer, TestLimitTokenCountFilter, TestAllAnalyzersHaveFactories]
   [junit4] Completed [100/280] on J0 in 0.90s, 1 test, 1 error <<< FAILURES!

[...truncated 548 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build.xml:471: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:2248: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/analysis/build.xml:106: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/analysis/build.xml:38: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/module-build.xml:58: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 280 suites (2 ignored), 1475 tests, 1 error, 1 ignored [seed: 150596469108C85C]

Total time: 69 minutes 32 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1012 - Still Failing

[Apache Jenkins Server <je...@builds.apache.org>]

Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1012/

All tests passed

Build Log:
[...truncated 4723 lines...]
   [junit4] JVM J2: stderr was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/backward-codecs/test/temp/junit4-J2-20151109_224734_067.syserr
   [junit4] >>> JVM J2: stderr (verbatim) ----
   [junit4] WARN: Unhandled exception in event serialization. -> java.security.PrivilegedActionException: java.io.IOException: No space left on device
   [junit4] 	at java.security.AccessController.doPrivileged(Native Method)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.events.Serializer.flushQueue(Serializer.java:96)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.events.Serializer.serialize(Serializer.java:81)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.slave.RunListenerEmitter.testAssumptionFailure(RunListenerEmitter.java:68)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.slave.NoExceptionRunListenerDecorator.testAssumptionFailure(NoExceptionRunListenerDecorator.java:63)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.slave.BeforeAfterRunListenerDecorator.testAssumptionFailure(BeforeAfterRunListenerDecorator.java:69)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.slave.OrderedRunNotifier$5.notifyListener(OrderedRunNotifier.java:146)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.slave.OrderedRunNotifier$SafeNotifier.run(OrderedRunNotifier.java:63)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.slave.OrderedRunNotifier.fireTestAssumptionFailed(OrderedRunNotifier.java:148)
   [junit4] 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$SubNotifier.fireTestAssumptionFailed(ThreadLeakControl.java:175)
   [junit4] 	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:879)
   [junit4] 	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:777)
   [junit4] 	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:811)
   [junit4] 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:822)
   [junit4] 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4] 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4] 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4] 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   [junit4] 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   [junit4] 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4] 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4] 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4] 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4] 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4] 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4] 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4] 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4] 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
   [junit4] 	at java.lang.Thread.run(Thread.java:745)
   [junit4] Caused by: java.io.IOException: No space left on device
   [junit4] 	at java.io.FileOutputStream.writeBytes(Native Method)
   [junit4] 	at java.io.FileOutputStream.write(FileOutputStream.java:345)
   [junit4] 	at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
   [junit4] 	at java.io.BufferedOutputStream.write(BufferedOutputStream.java:121)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.slave.EventsOutputStream.write(EventsOutputStream.java:28)
   [junit4] 	at sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:221)
   [junit4] 	at sun.nio.cs.StreamEncoder.implWrite(StreamEncoder.java:282)
   [junit4] 	at sun.nio.cs.StreamEncoder.write(StreamEncoder.java:125)
   [junit4] 	at sun.nio.cs.StreamEncoder.write(StreamEncoder.java:135)
   [junit4] 	at java.io.OutputStreamWriter.write(OutputStreamWriter.java:220)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.gson.stream.JsonWriter.string(JsonWriter.java:565)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.gson.stream.JsonWriter.value(JsonWriter.java:414)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.events.FailureEvent.serialize(FailureEvent.java:46)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.events.Serializer$2.run(Serializer.java:101)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.events.Serializer$2.run(Serializer.java:96)
   [junit4] 	... 29 more
   [junit4] <<< JVM J2: EOF ----

[...truncated 203 lines...]
   [junit4] ERROR: JVM J2 ended with an exception, command line: /x1/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.7/jre/bin/java -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/heapdumps -ea -esa -Dtests.prefix=tests -Dtests.seed=AB0208B9DF379D90 -Xmx512M -Dtests.iters= -Dtests.verbose=false -Dtests.infostream=false -Dtests.codec=random -Dtests.postingsformat=random -Dtests.docvaluesformat=random -Dtests.locale=random -Dtests.timezone=random -Dtests.directory=random -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.luceneMatchVersion=5.4.0 -Dtests.cleanthreads=perMethod -Djava.util.logging.config.file=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/tools/junit4/logging.properties -Dtests.nightly=true -Dtests.weekly=false -Dtests.monster=false -Dtests.slow=true -Dtests.asserts=true -Dtests.multiplier=2 -DtempDir=./temp -Djava.io.tmpdir=./temp -Djunit4.tempDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/backward-codecs/test/temp -Dcommon.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene -Dclover.db.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/clover/db -Djava.security.policy=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/tools/junit4/tests.policy -Dtests.LUCENE_VERSION=5.4.0 -Djetty.testMode=1 -Djetty.insecurerandom=1 -Dsolr.directoryFactory=org.apache.solr.core.MockDirectoryFactory -Djava.awt.headless=true -Djdk.map.althashing.threshold=0 -Djunit4.childvm.cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/backward-codecs/test/J2 -Djunit4.childvm.id=2 -Djunit4.childvm.count=3 -Dtests.leaveTemporary=false -Dtests.filterstacks=true -Djava.security.manager=org.apache.lucene.util.TestSecurityManager -Dfile.encoding=US-ASCII -classpath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/backward-codecs/classes/test:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/test-framework/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/codecs/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/core/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/test-framework/lib/junit-4.10.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/test-framework/lib/randomizedtesting-runner-2.2.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/backward-codecs/classes/java:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-launcher.jar:/x1/jenkins/.ant/lib/ivy-2.3.0.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-antlr.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-regexp.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-resolver.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-net.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jmf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-oro.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-xalan2.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-logging.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jsch.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit4.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-javamail.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jai.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jdepend.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bsf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-swing.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-netrexx.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bcel.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-testutil.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-log4j.jar:/x1/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.7/lib/tools.jar:/x1/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.2.0.jar com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/backward-codecs/test/temp/junit4-J2-20151109_224734_067.events @/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/backward-codecs/test/temp/junit4-J2-20151109_224734_067.suites -stdin
   [junit4] ERROR: JVM J2 ended with an exception: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1503)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:130)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:963)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:960)
   [junit4] 	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4] 	at java.lang.Thread.run(Thread.java:745)

BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build.xml:471: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:2248: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/module-build.xml:58: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: At least one slave process threw an exception, first: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.

Total time: 134 minutes 45 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1011 - Still Failing

[Apache Jenkins Server <je...@builds.apache.org>]

Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1011/

2 tests failed.
FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Error from server at http://127.0.0.1:43241/awholynewcollection_2: Expected mime type application/octet-stream but got text/html. <html> <head> <meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/> <title>Error 500 </title> </head> <body> <h2>HTTP ERROR: 500</h2> <p>Problem accessing /awholynewcollection_2/select. Reason: <pre>    {trace=java.lang.NullPointerException  at org.apache.solr.servlet.HttpSolrCall.getCoreByCollection(HttpSolrCall.java:786)  at org.apache.solr.servlet.HttpSolrCall.init(HttpSolrCall.java:270)  at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:415)  at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:220)  at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:179)  at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)  at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:109)  at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)  at org.eclipse.jetty.servlets.UserAgentFilter.doFilter(UserAgentFilter.java:83)  at org.eclipse.jetty.servlets.GzipFilter.doFilter(GzipFilter.java:364)  at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)  at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)  at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:221)  at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)  at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)  at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)  at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)  at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)  at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)  at org.eclipse.jetty.server.Server.handle(Server.java:499)  at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)  at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)  at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)  at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)  at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)  at java.lang.Thread.run(Thread.java:745) ,code=500}</pre></p> <hr /><i><small>Powered by Jetty://</small></i> </body> </html> 

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:43241/awholynewcollection_2: Expected mime type application/octet-stream but got text/html. <html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/>
<title>Error 500 </title>
</head>
<body>
<h2>HTTP ERROR: 500</h2>
<p>Problem accessing /awholynewcollection_2/select. Reason:
<pre>    {trace=java.lang.NullPointerException
	at org.apache.solr.servlet.HttpSolrCall.getCoreByCollection(HttpSolrCall.java:786)
	at org.apache.solr.servlet.HttpSolrCall.init(HttpSolrCall.java:270)
	at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:415)
	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:220)
	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:179)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
	at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:109)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
	at org.eclipse.jetty.servlets.UserAgentFilter.doFilter(UserAgentFilter.java:83)
	at org.eclipse.jetty.servlets.GzipFilter.doFilter(GzipFilter.java:364)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:221)
	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
	at org.eclipse.jetty.server.Server.handle(Server.java:499)
	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)
	at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
	at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)
	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
	at java.lang.Thread.run(Thread.java:745)
,code=500}</pre></p>
<hr /><i><small>Powered by Jetty://</small></i>
</body>
</html>

	at __randomizedtesting.SeedInfo.seed([4B08AF09266CA99E:C35C90D38890C466]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:543)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:150)
	at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:943)
	at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:958)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.waitForNon403or404or503(AbstractFullDistribZkTestBase.java:1754)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testCollectionsAPI(CollectionsAPIDistributedZkTest.java:665)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test(CollectionsAPIDistributedZkTest.java:167)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1660)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:866)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:902)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:916)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:963)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:938)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:875)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:777)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:811)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:822)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR

Error Message:
Captured an uncaught exception in thread: Thread[id=116988, name=coreZkRegister-4110-thread-1, state=RUNNABLE, group=TGRP-LeaderInitiatedRecoveryOnShardRestartTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=116988, name=coreZkRegister-4110-thread-1, state=RUNNABLE, group=TGRP-LeaderInitiatedRecoveryOnShardRestartTest]
	at __randomizedtesting.SeedInfo.seed([4B08AF09266CA99E:9582C1741DCECB6D]:0)
Caused by: java.lang.AssertionError
	at __randomizedtesting.SeedInfo.seed([4B08AF09266CA99E]:0)
	at org.apache.solr.cloud.ZkController.updateLeaderInitiatedRecoveryState(ZkController.java:2133)
	at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:452)
	at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)
	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)
	at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:346)
	at org.apache.solr.cloud.ZkController.joinElection(ZkController.java:1113)
	at org.apache.solr.cloud.ZkController.register(ZkController.java:926)
	at org.apache.solr.cloud.ZkController.register(ZkController.java:881)
	at org.apache.solr.core.ZkContainer$2.run(ZkContainer.java:183)
	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 10394 lines...]
   [junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_4B08AF09266CA99E-001/init-core-data-001
   [junit4]   2> 562874 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[4B08AF09266CA99E]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false)
   [junit4]   2> 562874 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[4B08AF09266CA99E]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   2> 562878 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 562878 INFO  (Thread-3802) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 562878 INFO  (Thread-3802) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 562978 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.ZkTestServer start zk server on port:41093
   [junit4]   2> 562979 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 562979 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 562981 INFO  (zkCallback-414-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5ea23c19 name:ZooKeeperConnection Watcher:127.0.0.1:41093 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 562982 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 562982 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 562982 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 562985 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 562985 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 562986 INFO  (zkCallback-415-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2607a393 name:ZooKeeperConnection Watcher:127.0.0.1:41093/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 562987 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 562987 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 562987 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 562989 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 562990 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 562992 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 562993 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 562993 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 562996 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 562996 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 562998 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 562998 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 563000 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 563000 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 563002 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 563002 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 563004 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 563004 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 563005 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 563005 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 563007 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 563007 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 563008 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 563009 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 563010 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 563010 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 563012 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 563012 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 563014 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 563014 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 563015 INFO  (zkCallback-416-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@3faec4e9 name:ZooKeeperConnection Watcher:127.0.0.1:41093 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 563016 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 563016 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 563016 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 563017 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 563018 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 563019 INFO  (zkCallback-417-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@54fc1ab2 name:ZooKeeperConnection Watcher:127.0.0.1:41093/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 563019 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 563019 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 563020 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf2/solrconfig.xml
   [junit4]   2> 563020 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/solrconfig.xml
   [junit4]   2> 563022 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf2/schema.xml
   [junit4]   2> 563022 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/schema.xml
   [junit4]   2> 563024 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf2/enumsConfig.xml
   [junit4]   2> 563024 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/enumsConfig.xml
   [junit4]   2> 563025 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf2/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 563025 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 563027 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf2/stopwords.txt
   [junit4]   2> 563027 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/stopwords.txt
   [junit4]   2> 563028 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf2/protwords.txt
   [junit4]   2> 563029 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/protwords.txt
   [junit4]   2> 563030 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf2/currency.xml
   [junit4]   2> 563030 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/currency.xml
   [junit4]   2> 563032 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf2/open-exchange-rates.json
   [junit4]   2> 563032 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/open-exchange-rates.json
   [junit4]   2> 563033 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf2/mapping-ISOLatin1Accent.txt
   [junit4]   2> 563033 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/mapping-ISOLatin1Accent.txt
   [junit4]   2> 563035 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf2/old_synonyms.txt
   [junit4]   2> 563035 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/old_synonyms.txt
   [junit4]   2> 563036 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf2/synonyms.txt
   [junit4]   2> 563037 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/synonyms.txt
   [junit4]   2> 563038 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/elevate.xml to /configs/conf2/elevate.xml
   [junit4]   2> 563038 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/elevate.xml
   [junit4]   2> 563340 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_4B08AF09266CA99E-001/control-001/cores/collection1
   [junit4]   2> 563342 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 563343 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@3f263574{/,null,AVAILABLE}
   [junit4]   2> 563347 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.e.j.s.ServerConnector Started ServerConnector@125b11d8{HTTP/1.1}{127.0.0.1:41854}
   [junit4]   2> 563347 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.e.j.s.Server Started @566074ms
   [junit4]   2> 563347 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_4B08AF09266CA99E-001/control-001/cores, hostPort=41854, hostContext=/, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_4B08AF09266CA99E-001/tempDir-001/control/data}
   [junit4]   2> 563347 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@3827de24
   [junit4]   2> 563347 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_4B08AF09266CA99E-001/control-001/'
   [junit4]   2> 563347 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 563347 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 563367 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 563367 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 563369 INFO  (zkCallback-418-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@37862268 name:ZooKeeperConnection Watcher:127.0.0.1:41093/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 563369 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 563369 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 563370 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 563370 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_4B08AF09266CA99E-001/control-001/solr.xml
   [junit4]   2> 563400 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_4B08AF09266CA99E-001/control-001/cores
   [junit4]   2> 563400 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.CoreContainer New CoreContainer 1772405105
   [junit4]   2> 563401 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_4B08AF09266CA99E-001/control-001/]
   [junit4]   2> 563401 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_4B08AF09266CA99E-001/control-001/lib
   [junit4]   2> 563401 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_4B08AF09266CA99E-001/control-001/lib).
   [junit4]   2> 563408 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 563409 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 563410 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 563410 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 563410 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:41093/solr
   [junit4]   2> 563410 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 563410 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 563411 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 563412 INFO  (zkCallback-420-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2f60ed9f name:ZooKeeperConnection Watcher:127.0.0.1:41093 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 563412 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 563413 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 563414 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [n:127.0.0.1:41854_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 563416 INFO  (zkCallback-421-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7b901df6 name:ZooKeeperConnection Watcher:127.0.0.1:41093/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 563416 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [n:127.0.0.1:41854_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 563417 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [n:127.0.0.1:41854_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 563419 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [n:127.0.0.1:41854_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 563422 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [n:127.0.0.1:41854_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 563424 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [n:127.0.0.1:41854_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 563427 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [n:127.0.0.1:41854_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 563429 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [n:127.0.0.1:41854_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 563430 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [n:127.0.0.1:41854_    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 563432 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [n:127.0.0.1:41854_    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 563433 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [n:127.0.0.1:41854_    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 563434 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [n:127.0.0.1:41854_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:41854_
   [junit4]   2> 563434 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [n:127.0.0.1:41854_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:41854_
   [junit4]   2> 563436 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [n:127.0.0.1:41854_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 563437 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [n:127.0.0.1:41854_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 563438 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [n:127.0.0.1:41854_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 563439 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [n:127.0.0.1:41854_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94832189490528262-127.0.0.1:41854_-n_0000000000
   [junit4]   2> 563439 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [n:127.0.0.1:41854_    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:41854_
   [junit4]   2> 563439 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [n:127.0.0.1:41854_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 563441 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [n:127.0.0.1:41854_    ] o.a.s.c.Overseer Overseer (id=94832189490528262-127.0.0.1:41854_-n_0000000000) starting
   [junit4]   2> 563442 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [n:127.0.0.1:41854_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 563448 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [n:127.0.0.1:41854_    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 563449 INFO  (OverseerCollectionConfigSetProcessor-94832189490528262-127.0.0.1:41854_-n_0000000000) [n:127.0.0.1:41854_    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 563449 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [n:127.0.0.1:41854_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 563449 INFO  (OverseerStateUpdate-94832189490528262-127.0.0.1:41854_-n_0000000000) [n:127.0.0.1:41854_    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 563465 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [n:127.0.0.1:41854_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 563466 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [n:127.0.0.1:41854_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 563466 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [n:127.0.0.1:41854_    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_4B08AF09266CA99E-001/control-001/cores
   [junit4]   2> 563467 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [n:127.0.0.1:41854_    ] o.a.s.c.SolrCore Created CoreDescriptor: {configSetProperties=configsetprops.json, collection=control_collection, dataDir=data/, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_4B08AF09266CA99E-001/control-001/cores/collection1/, schema=schema.xml, shard=, coreNodeName=, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_4B08AF09266CA99E-001/control-001/cores/collection1, config=solrconfig.xml, name=collection1, transient=false, loadOnStartup=true}
   [junit4]   2> 563467 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [n:127.0.0.1:41854_    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_4B08AF09266CA99E-001/control-001/cores/collection1/
   [junit4]   2> 563467 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [n:127.0.0.1:41854_    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 563468 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [n:127.0.0.1:41854_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 563468 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [n:127.0.0.1:41854_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 563468 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 563468 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 563471 INFO  (OverseerStateUpdate-94832189490528262-127.0.0.1:41854_-n_0000000000) [n:127.0.0.1:41854_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "roles":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:41854",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "node_name":"127.0.0.1:41854_",
   [junit4]   2>   "core":"collection1"} current state version: 0
   [junit4]   2> 563471 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 563471 INFO  (OverseerStateUpdate-94832189490528262-127.0.0.1:41854_-n_0000000000) [n:127.0.0.1:41854_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "roles":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:41854",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "node_name":"127.0.0.1:41854_",
   [junit4]   2>   "core":"collection1"}
   [junit4]   2> 563471 INFO  (OverseerStateUpdate-94832189490528262-127.0.0.1:41854_-n_0000000000) [n:127.0.0.1:41854_    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 563472 INFO  (OverseerStateUpdate-94832189490528262-127.0.0.1:41854_-n_0000000000) [n:127.0.0.1:41854_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 563473 INFO  (zkCallback-421-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 564471 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 564471 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 564472 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 564472 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 564473 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 564473 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_4B08AF09266CA99E-001/control-001/cores/collection1/'
   [junit4]   2> 564473 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 564473 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 564488 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 564499 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 564516 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 564550 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 564572 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 564580 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 565031 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 565039 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 565045 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 565054 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 565057 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 565060 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 565061 WARN  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 565061 WARN  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 565061 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 565061 WARN  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 565061 WARN  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 565062 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 565062 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 565062 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 565063 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_4B08AF09266CA99E-001/control-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 565063 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@28ee7d61
   [junit4]   2> 565063 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_4B08AF09266CA99E-001/control-001/cores/collection1/data
   [junit4]   2> 565063 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_4B08AF09266CA99E-001/control-001/cores/collection1/data/index/
   [junit4]   2> 565063 WARN  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_4B08AF09266CA99E-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 565064 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_4B08AF09266CA99E-001/control-001/cores/collection1/data/index
   [junit4]   2> 565064 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=30, maxMergeAtOnceExplicit=49, maxMergedSegmentMB=19.6376953125, floorSegmentMB=1.408203125, forceMergeDeletesPctAllowed=9.387081741303945, segmentsPerTier=17.0, maxCFSSegmentSizeMB=1.3232421875, noCFSRatio=0.812134224251013
   [junit4]   2> 565065 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(NRTCachingDirectory(RAMDirectory@3be62ea0 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@4ff2919; maxCacheMB=0.8740234375 maxMergeSizeMB=0.294921875)),segFN=segments_1,generation=1}
   [junit4]   2> 565065 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 565069 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 565069 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 565070 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 565070 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 565070 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 565070 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 565070 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 565070 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 565070 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 565071 WARN  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
   [junit4]   2> 565074 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_4B08AF09266CA99E-001/control-001/cores/collection1/data/
   [junit4]   2> 565074 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_4B08AF09266CA99E-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 565076 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 565078 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 565079 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 565080 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 565084 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /update/csv,/get,/replication,/config,/schema,/admin/file,standard,/admin/threads,/admin/system,/admin/plugins,/admin/luke,/admin/ping,/admin/properties,/update/json,/update/json/docs,/admin/logging,/admin/mbeans,/dump,/admin/segments,/update
   [junit4]   2> 565085 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 565086 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 565086 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 565087 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 565087 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 565087 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=34, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.20798483476383636]
   [junit4]   2> 565087 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(NRTCachingDirectory(RAMDirectory@3be62ea0 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@4ff2919; maxCacheMB=0.8740234375 maxMergeSizeMB=0.294921875)),segFN=segments_1,generation=1}
   [junit4]   2> 565088 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 565088 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@71ef6bc0[collection1] main
   [junit4]   2> 565088 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 565088 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 565089 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 565089 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 565089 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 565089 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 565089 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 565089 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 565089 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 565090 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 565090 INFO  (searcherExecutor-1629-thread-1-processing-c:control_collection s:shard1 r:core_node1 x:collection1 n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@71ef6bc0[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 565090 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 565090 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 565090 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 565090 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1517315034164756480
   [junit4]   2> 565092 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 2.0ms to seed version buckets with highest version 1517315034164756480
   [junit4]   2> 565093 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 565093 INFO  (coreLoadExecutor-1628-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 565101 INFO  (coreZkRegister-1622-thread-1-processing-c:control_collection s:shard1 r:core_node1 x:collection1 n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:41854 collection:control_collection shard:shard1
   [junit4]   2> 565103 INFO  (coreZkRegister-1622-thread-1-processing-c:control_collection s:shard1 r:core_node1 x:collection1 n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 565106 INFO  (coreZkRegister-1622-thread-1-processing-c:control_collection s:shard1 r:core_node1 x:collection1 n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94832189490528262-core_node1-n_0000000000
   [junit4]   2> 565107 INFO  (coreZkRegister-1622-thread-1-processing-c:control_collection s:shard1 r:core_node1 x:collection1 n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 565108 INFO  (coreZkRegister-1622-thread-1-processing-c:control_collection s:shard1 r:core_node1 x:collection1 n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 565108 INFO  (OverseerStateUpdate-94832189490528262-127.0.0.1:41854_-n_0000000000) [n:127.0.0.1:41854_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 565109 INFO  (coreZkRegister-1622-thread-1-processing-c:control_collection s:shard1 r:core_node1 x:collection1 n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 565109 INFO  (coreZkRegister-1622-thread-1-processing-c:control_collection s:shard1 r:core_node1 x:collection1 n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:41854/collection1/
   [junit4]   2> 565109 INFO  (coreZkRegister-1622-thread-1-processing-c:control_collection s:shard1 r:core_node1 x:collection1 n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 565110 INFO  (coreZkRegister-1622-thread-1-processing-c:control_collection s:shard1 r:core_node1 x:collection1 n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:41854/collection1/ has no replicas
   [junit4]   2> 565110 INFO  (coreZkRegister-1622-thread-1-processing-c:control_collection s:shard1 r:core_node1 x:collection1 n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:41854/collection1/ shard1
   [junit4]   2> 565110 INFO  (zkCallback-421-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 565120 INFO  (coreZkRegister-1622-thread-1-processing-c:control_collection s:shard1 r:core_node1 x:collection1 n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 565122 INFO  (coreZkRegister-1622-thread-1-processing-c:control_collection s:shard1 r:core_node1 x:collection1 n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 565124 INFO  (OverseerStateUpdate-94832189490528262-127.0.0.1:41854_-n_0000000000) [n:127.0.0.1:41854_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:41854",
   [junit4]   2>   "node_name":"127.0.0.1:41854_",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "core":"collection1"} current state version: 2
   [junit4]   2> 565125 INFO  (OverseerStateUpdate-94832189490528262-127.0.0.1:41854_-n_0000000000) [n:127.0.0.1:41854_    ] o.a.s.c.o.ReplicaMutator Update state numShards=null message={
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:41854",
   [junit4]   2>   "node_name":"127.0.0.1:41854_",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "core":"collection1"}
   [junit4]   2> 565142 INFO  (OverseerStateUpdate-94832189490528262-127.0.0.1:41854_-n_0000000000) [n:127.0.0.1:41854_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:41854",
   [junit4]   2>   "core":"collection1"} current state version: 2
   [junit4]   2> 565256 INFO  (zkCallback-421-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 565297 INFO  (coreZkRegister-1622-thread-1-processing-c:control_collection s:shard1 r:core_node1 x:collection1 n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:41854/collection1/ and leader is http://127.0.0.1:41854/collection1/
   [junit4]   2> 565297 INFO  (coreZkRegister-1622-thread-1-processing-c:control_collection s:shard1 r:core_node1 x:collection1 n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:41854
   [junit4]   2> 565297 INFO  (coreZkRegister-1622-thread-1-processing-c:control_collection s:shard1 r:core_node1 x:collection1 n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 565297 INFO  (coreZkRegister-1622-thread-1-processing-c:control_collection s:shard1 r:core_node1 x:collection1 n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 565297 INFO  (coreZkRegister-1622-thread-1-processing-c:control_collection s:shard1 r:core_node1 x:collection1 n:127.0.0.1:41854_) [n:127.0.0.1:41854_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 565298 INFO  (OverseerStateUpdate-94832189490528262-127.0.0.1:41854_-n_0000000000) [n:127.0.0.1:41854_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "roles":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "base_url":"http://127.0.0.1:41854",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "node_name":"127.0.0.1:41854_",
   [junit4]   2>   "core":"collection1"} current state version: 3
   [junit4]   2> 565299 INFO  (OverseerStateUpdate-94832189490528262-127.0.0.1:41854_-n_0000000000) [n:127.0.0.1:41854_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "roles":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "base_url":"http://127.0.0.1:41854",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "node_name":"127.0.0.1:41854_",
   [junit4]   2>   "core":"collection1"}
   [junit4]   2> 565402 INFO  (zkCallback-421-thread-1-processing-n:127.0.0.1:41854_) [n:127.0.0.1:41854_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 565470 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 565471 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 565472 INFO  (zkCallback-423-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2df15bb2 name:ZooKeeperConnection Watcher:127.0.0.1:41093/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 565472 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 565472 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 565472 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 565474 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 565772 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_4B08AF09266CA99E-001/shard-1-001/cores/collection1
   [junit4]   2> 565773 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_4B08AF09266CA99E-001/shard-1-001
   [junit4]   2> 565774 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 565776 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@e8ca006{/,null,AVAILABLE}
   [junit4]   2> 565776 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.e.j.s.ServerConnector Started ServerConnector@3591ef75{HTTP/1.1}{127.0.0.1:60699}
   [junit4]   2> 565776 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.e.j.s.Server Started @568503ms
   [junit4]   2> 565777 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solrconfig=solrconfig.xml, hostContext=/, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_4B08AF09266CA99E-001/tempDir-001/jetty1, hostPort=60699, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_4B08AF09266CA99E-001/shard-1-001/cores}
   [junit4]   2> 565777 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@3827de24
   [junit4]   2> 565777 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_4B08AF09266CA99E-001/shard-1-001/'
   [junit4]   2> 565777 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 565777 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 565793 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 565794 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 565795 INFO  (zkCallback-424-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1e4acdf0 name:ZooKeeperConnection Watcher:127.0.0.1:41093/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 565795 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 565795 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 565796 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 565796 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_4B08AF09266CA99E-001/shard-1-001/solr.xml
   [junit4]   2> 565810 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_4B08AF09266CA99E-001/shard-1-001/cores
   [junit4]   2> 565810 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.CoreContainer New CoreContainer 1192663454
   [junit4]   2> 565810 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_4B08AF09266CA99E-001/shard-1-001/]
   [junit4]   2> 565810 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_4B08AF09266CA99E-001/shard-1-001/lib
   [junit4]   2> 565810 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_4B08AF09266CA99E-001/shard-1-001/lib).
   [junit4]   2> 565818 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 565819 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 565819 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 565820 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 565820 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:41093/solr
   [junit4]   2> 565820 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 565820 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 565820 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 565822 INFO  (zkCallback-426-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@f7174fb name:ZooKeeperConnection Watcher:127.0.0.1:41093 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 565822 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 565823 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 565824 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[4B08AF09266CA99E]) [n:127.0.0.1:60699_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 565825 INFO  (zkCallback-427-thread-1-processing-n:127.0.0.1:60699_) [n:127.0.0.1:60699_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.c

[...truncated too long message...]

at org.apache.solr.common.cloud.SolrZkClient$2.execute(SolrZkClient.java:241)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:61)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.delete(SolrZkClient.java:241)
   [junit4]   2> 	at org.apache.solr.cloud.ElectionContext.cancelElection(ElectionContext.java:86)
   [junit4]   2> 	at org.apache.solr.cloud.ShardLeaderElectionContextBase.cancelElection(ElectionContext.java:169)
   [junit4]   2> 	at org.apache.solr.cloud.ShardLeaderElectionContext.cancelElection(ElectionContext.java:286)
   [junit4]   2> 	at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:468)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.access$200(LeaderElector.java:58)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector$ElectionWatcher.process(LeaderElector.java:388)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$3$1.run(SolrZkClient.java:264)
   [junit4]   2> 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
   [junit4]   2> 	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
   [junit4]   2> 	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 2416914 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[4B08AF09266CA99E]) [    ] o.e.j.s.ServerConnector Stopped ServerConnector@59a95e4c{HTTP/1.1}{127.0.0.1:58129}
   [junit4]   2> 2416914 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[4B08AF09266CA99E]) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@18fa503b{/,null,UNAVAILABLE}
   [junit4]   2> 2416917 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[4B08AF09266CA99E]) [n:127.0.0.1:58129_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.ChaosMonkey monkey: stop shard! 46684
   [junit4]   2> 2416918 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[4B08AF09266CA99E]) [n:127.0.0.1:58129_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.ZkTestServer connecting to 127.0.0.1:59074 59074
   [junit4]   2> 2417154 INFO  (Thread-109844) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:59074 59074
   [junit4]   2> 2417154 WARN  (Thread-109844) [    ] o.a.s.c.ZkTestServer Watch limit violations: 
   [junit4]   2> Maximum concurrent create/delete watches above limit:
   [junit4]   2> 
   [junit4]   2> 	6	/solr/aliases.json
   [junit4]   2> 	5	/solr/configs/conf1
   [junit4]   2> 	5	/solr/security.json
   [junit4]   2> 
   [junit4]   2> Maximum concurrent data watches above limit:
   [junit4]   2> 
   [junit4]   2> 	18	/solr/collections/collection1/state.json
   [junit4]   2> 	6	/solr/clusterstate.json
   [junit4]   2> 	4	/solr/collections/all_in_lir/state.json
   [junit4]   2> 	3	/solr/overseer_elect/election/94832308381155332-127.0.0.1:46684_-n_0000000000
   [junit4]   2> 
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2> 
   [junit4]   2> 	6	/solr/collections
   [junit4]   2> 	4	/solr/live_nodes
   [junit4]   2> 	3	/solr/overseer/collection-queue-work
   [junit4]   2> 	2	/solr/overseer/queue
   [junit4]   2> 
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=LeaderInitiatedRecoveryOnShardRestartTest -Dtests.method=testRestartWithAllInLIR -Dtests.seed=4B08AF09266CA99E -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=ar_QA -Dtests.timezone=Etc/Universal -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
   [junit4] ERROR   40.2s J2 | LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR <<<
   [junit4]    > Throwable #1: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=116988, name=coreZkRegister-4110-thread-1, state=RUNNABLE, group=TGRP-LeaderInitiatedRecoveryOnShardRestartTest]
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([4B08AF09266CA99E:9582C1741DCECB6D]:0)
   [junit4]    > Caused by: java.lang.AssertionError
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([4B08AF09266CA99E]:0)
   [junit4]    > 	at org.apache.solr.cloud.ZkController.updateLeaderInitiatedRecoveryState(ZkController.java:2133)
   [junit4]    > 	at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:452)
   [junit4]    > 	at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)
   [junit4]    > 	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)
   [junit4]    > 	at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:346)
   [junit4]    > 	at org.apache.solr.cloud.ZkController.joinElection(ZkController.java:1113)
   [junit4]    > 	at org.apache.solr.cloud.ZkController.register(ZkController.java:926)
   [junit4]    > 	at org.apache.solr.cloud.ZkController.register(ZkController.java:881)
   [junit4]    > 	at org.apache.solr.core.ZkContainer$2.run(ZkContainer.java:183)
   [junit4]    > 	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)
   [junit4]    > 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   [junit4]    > 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 2417159 INFO  (SUITE-LeaderInitiatedRecoveryOnShardRestartTest-seed#[4B08AF09266CA99E]-worker) [n:127.0.0.1:58129_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene54): {}, docValues:{}, sim=DefaultSimilarity, locale=ar_QA, timezone=Etc/Universal
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_80 (64-bit)/cpus=4,threads=1,free=275076840,total=520093696
   [junit4]   2> NOTE: All tests run in this JVM: [DistributedExpandComponentTest, TestHashQParserPlugin, SolrCoreCheckLockOnStartupTest, ClusterStateTest, TestSolrQueryParserResource, TestPHPSerializedResponseWriter, ChangedSchemaMergeTest, TestReload, TermVectorComponentDistributedTest, AsyncCallRequestStatusResponseTest, TestRuleBasedAuthorizationPlugin, RollingRestartTest, LukeRequestHandlerTest, TestSchemaVersionResource, TestManagedSchemaFieldTypeResource, StatsComponentTest, SecurityConfHandlerTest, BufferStoreTest, SliceStateTest, TestBlendedInfixSuggestions, TestRestManager, TestSolrDynamicMBean, TestStressRecovery, SuggesterTSTTest, CursorPagingTest, TestIndexSearcher, FieldAnalysisRequestHandlerTest, IgnoreCommitOptimizeUpdateProcessorFactoryTest, DocExpirationUpdateProcessorFactoryTest, TestSearchPerf, HdfsBasicDistributedZkTest, TestDefaultSearchFieldResource, TestSmileRequest, TestRemoveLastDynamicCopyField, TestCollationField, HdfsLockFactoryTest, UpdateParamsTest, DistributedFacetPivotLongTailTest, TestIntervalFaceting, TestDynamicFieldCollectionResource, BJQParserTest, QueryResultKeyTest, TestStressReorder, TestReloadAndDeleteDocs, TestSolrQueryParserDefaultOperatorResource, TestSort, TestCopyFieldCollectionResource, TestFuzzyAnalyzedSuggestions, TestConfigSetImmutable, AssignTest, TestCloudManagedSchema, SimpleFacetsTest, TestReversedWildcardFilterFactory, RegexBoostProcessorTest, RequestHandlersTest, SearchHandlerTest, TestCoreDiscovery, TestSolrQueryParser, TestDefaultStatsCache, TestSweetSpotSimilarityFactory, TestFieldTypeResource, TestGroupingSearch, TestPerFieldSimilarity, TestRecovery, CoreAdminCreateDiscoverTest, CoreAdminRequestStatusTest, TestUtils, TestOverriddenPrefixQueryForCustomFieldType, CachingDirectoryFactoryTest, TestIndexingPerformance, ScriptEngineTest, TestSolrIndexConfig, ShowFileRequestHandlerTest, TestSchemaManager, TestCloudManagedSchemaConcurrent, LeaderInitiatedRecoveryOnShardRestartTest]
   [junit4] Completed [366/545] on J2 in 40.75s, 1 test, 1 error <<< FAILURES!

[...truncated 595 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 545 suites (7 ignored), 2158 tests, 2 errors, 32 ignored (18 assumptions) [seed: 4B08AF09266CA99E]

Total time: 193 minutes 23 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1010 - Still Failing

[Apache Jenkins Server <je...@builds.apache.org>]

Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1010/

3 tests failed.
FAILED:  org.apache.solr.cloud.ForceLeaderTest.testReplicasInLIRNoLeader

Error Message:
Expected that the LIR state would've been down by now expected:<down> but was:<recovering>

Stack Trace:
java.lang.AssertionError: Expected that the LIR state would've been down by now expected:<down> but was:<recovering>
	at __randomizedtesting.SeedInfo.seed([E96E92B971ECA2A3:FF9A679486E5BC2]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.failNotEquals(Assert.java:647)
	at org.junit.Assert.assertEquals(Assert.java:128)
	at org.apache.solr.cloud.ForceLeaderTest.putNonLeadersIntoLIR(ForceLeaderTest.java:386)
	at org.apache.solr.cloud.ForceLeaderTest.testReplicasInLIRNoLeader(ForceLeaderTest.java:92)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1660)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:866)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:902)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:916)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:963)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:938)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:875)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:777)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:811)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:822)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.cloud.OverseerTest.testOverseerStatsReset

Error Message:
expected:<0> but was:<1>

Stack Trace:
java.lang.AssertionError: expected:<0> but was:<1>
	at __randomizedtesting.SeedInfo.seed([E96E92B971ECA2A3:423A7585E43601AD]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.failNotEquals(Assert.java:647)
	at org.junit.Assert.assertEquals(Assert.java:128)
	at org.junit.Assert.assertEquals(Assert.java:472)
	at org.junit.Assert.assertEquals(Assert.java:456)
	at org.apache.solr.cloud.OverseerTest.testOverseerStatsReset(OverseerTest.java:732)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1660)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:866)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:902)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:916)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:875)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:777)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:811)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:822)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR

Error Message:
Captured an uncaught exception in thread: Thread[id=160214, name=coreZkRegister-966-thread-1, state=RUNNABLE, group=TGRP-LeaderInitiatedRecoveryOnShardRestartTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=160214, name=coreZkRegister-966-thread-1, state=RUNNABLE, group=TGRP-LeaderInitiatedRecoveryOnShardRestartTest]
	at __randomizedtesting.SeedInfo.seed([E96E92B971ECA2A3:37E4FCC44A4EC050]:0)
Caused by: java.lang.AssertionError
	at __randomizedtesting.SeedInfo.seed([E96E92B971ECA2A3]:0)
	at org.apache.solr.cloud.ZkController.updateLeaderInitiatedRecoveryState(ZkController.java:2133)
	at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:452)
	at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)
	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)
	at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:346)
	at org.apache.solr.cloud.ZkController.joinElection(ZkController.java:1113)
	at org.apache.solr.cloud.ZkController.register(ZkController.java:926)
	at org.apache.solr.cloud.ZkController.register(ZkController.java:881)
	at org.apache.solr.core.ZkContainer$2.run(ZkContainer.java:183)
	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 10620 lines...]
   [junit4] Suite: org.apache.solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_E96E92B971ECA2A3-001/init-core-data-001
   [junit4]   2> 1477012 INFO  (SUITE-LeaderInitiatedRecoveryOnShardRestartTest-seed#[E96E92B971ECA2A3]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   2> 1477022 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 1477023 INFO  (Thread-158410) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1477023 INFO  (Thread-158410) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 1477123 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.ZkTestServer start zk server on port:36301
   [junit4]   2> 1477123 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1477125 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1477140 INFO  (zkCallback-298-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2098723 name:ZooKeeperConnection Watcher:127.0.0.1:36301 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1477140 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1477141 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1477141 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 1477163 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x150e44979190000, likely client has closed socket
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 1477164 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1477172 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1477182 INFO  (zkCallback-299-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1469e01e name:ZooKeeperConnection Watcher:127.0.0.1:36301/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1477182 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1477183 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1477183 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 1477188 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 1477189 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 1477192 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 1477194 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 1477194 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 1477199 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 1477199 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 1477205 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1477205 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1477210 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 1477211 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 1477214 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 1477214 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 1477217 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 1477217 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 1477222 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 1477222 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 1477225 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1477226 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1477229 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1477229 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1477232 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 1477232 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 1477238 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 1477238 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 1477590 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_E96E92B971ECA2A3-001/control-001/cores/collection1
   [junit4]   2> 1477592 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 1477595 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@6abd1abc{/,null,AVAILABLE}
   [junit4]   2> 1477596 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.e.j.s.ServerConnector Started ServerConnector@7b470ba2{HTTP/1.1}{127.0.0.1:35300}
   [junit4]   2> 1477596 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.e.j.s.Server Started @1480714ms
   [junit4]   2> 1477596 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_E96E92B971ECA2A3-001/control-001/cores, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_E96E92B971ECA2A3-001/tempDir-001/control/data, hostContext=/, hostPort=35300}
   [junit4]   2> 1477596 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@3827de24
   [junit4]   2> 1477596 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_E96E92B971ECA2A3-001/control-001/'
   [junit4]   2> 1477596 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1477596 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1477614 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1477615 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1477616 INFO  (zkCallback-300-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@42e5e1f7 name:ZooKeeperConnection Watcher:127.0.0.1:36301/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1477616 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1477616 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1477617 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1477617 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_E96E92B971ECA2A3-001/control-001/solr.xml
   [junit4]   2> 1477632 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_E96E92B971ECA2A3-001/control-001/cores
   [junit4]   2> 1477632 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.CoreContainer New CoreContainer 1538089409
   [junit4]   2> 1477632 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_E96E92B971ECA2A3-001/control-001/]
   [junit4]   2> 1477632 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_E96E92B971ECA2A3-001/control-001/lib
   [junit4]   2> 1477632 WARN  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_E96E92B971ECA2A3-001/control-001/lib).
   [junit4]   2> 1477642 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1477643 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1477643 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1477643 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1477644 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:36301/solr
   [junit4]   2> 1477644 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 1477644 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1477652 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1477663 INFO  (zkCallback-302-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5908822f name:ZooKeeperConnection Watcher:127.0.0.1:36301 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1477664 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1477665 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1477670 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [n:127.0.0.1:35300_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1477675 INFO  (zkCallback-303-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@361956ce name:ZooKeeperConnection Watcher:127.0.0.1:36301/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1477675 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [n:127.0.0.1:35300_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1477677 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [n:127.0.0.1:35300_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 1477682 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [n:127.0.0.1:35300_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 1477691 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [n:127.0.0.1:35300_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 1477703 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [n:127.0.0.1:35300_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 1477710 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [n:127.0.0.1:35300_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 1477713 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [n:127.0.0.1:35300_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 1477715 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [n:127.0.0.1:35300_    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 1477716 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [n:127.0.0.1:35300_    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 1477718 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [n:127.0.0.1:35300_    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 1477720 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [n:127.0.0.1:35300_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:35300_
   [junit4]   2> 1477721 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [n:127.0.0.1:35300_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:35300_
   [junit4]   2> 1477723 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [n:127.0.0.1:35300_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 1477734 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [n:127.0.0.1:35300_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 1477744 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [n:127.0.0.1:35300_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1477744 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [n:127.0.0.1:35300_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94826596390207492-127.0.0.1:35300_-n_0000000000
   [junit4]   2> 1477745 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [n:127.0.0.1:35300_    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:35300_
   [junit4]   2> 1477745 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [n:127.0.0.1:35300_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 1477746 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [n:127.0.0.1:35300_    ] o.a.s.c.Overseer Overseer (id=94826596390207492-127.0.0.1:35300_-n_0000000000) starting
   [junit4]   2> 1477750 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [n:127.0.0.1:35300_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 1477766 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [n:127.0.0.1:35300_    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 1477767 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [n:127.0.0.1:35300_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1477767 INFO  (OverseerCollectionConfigSetProcessor-94826596390207492-127.0.0.1:35300_-n_0000000000) [n:127.0.0.1:35300_    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 1477767 INFO  (OverseerStateUpdate-94826596390207492-127.0.0.1:35300_-n_0000000000) [n:127.0.0.1:35300_    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 1477806 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [n:127.0.0.1:35300_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1477806 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [n:127.0.0.1:35300_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1477808 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [n:127.0.0.1:35300_    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_E96E92B971ECA2A3-001/control-001/cores
   [junit4]   2> 1477808 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [n:127.0.0.1:35300_    ] o.a.s.c.SolrCore Created CoreDescriptor: {coreNodeName=, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_E96E92B971ECA2A3-001/control-001/cores/collection1/, collection=control_collection, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_E96E92B971ECA2A3-001/control-001/cores/collection1, schema=schema.xml, configSetProperties=configsetprops.json, shard=, transient=false, dataDir=data/, config=solrconfig.xml, name=collection1, loadOnStartup=true}
   [junit4]   2> 1477809 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [n:127.0.0.1:35300_    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_E96E92B971ECA2A3-001/control-001/cores/collection1/
   [junit4]   2> 1477809 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [n:127.0.0.1:35300_    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 1477811 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [n:127.0.0.1:35300_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 1477811 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [n:127.0.0.1:35300_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1477813 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 1477813 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1477818 INFO  (OverseerStateUpdate-94826596390207492-127.0.0.1:35300_-n_0000000000) [n:127.0.0.1:35300_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "roles":null,
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "base_url":"http://127.0.0.1:35300",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "node_name":"127.0.0.1:35300_",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"down"} current state version: 0
   [junit4]   2> 1477818 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 1477819 INFO  (OverseerStateUpdate-94826596390207492-127.0.0.1:35300_-n_0000000000) [n:127.0.0.1:35300_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "roles":null,
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "base_url":"http://127.0.0.1:35300",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "node_name":"127.0.0.1:35300_",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"down"}
   [junit4]   2> 1477820 INFO  (OverseerStateUpdate-94826596390207492-127.0.0.1:35300_-n_0000000000) [n:127.0.0.1:35300_    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 1477821 INFO  (OverseerStateUpdate-94826596390207492-127.0.0.1:35300_-n_0000000000) [n:127.0.0.1:35300_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 1477825 INFO  (zkCallback-303-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1478819 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1478819 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 1478820 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 1478820 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 1478820 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1478820 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_E96E92B971ECA2A3-001/control-001/cores/collection1/'
   [junit4]   2> 1478821 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1478821 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1478840 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 1478848 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 1478872 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 1478925 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1478926 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 1478934 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1479221 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 1479241 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 1479242 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 1479258 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1479262 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1479266 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1479267 WARN  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 1479267 WARN  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 1479267 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1479268 WARN  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 1479268 WARN  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 1479269 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 1479269 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 1479269 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 1479269 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_E96E92B971ECA2A3-001/control-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 1479269 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@44e8518c
   [junit4]   2> 1479270 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_E96E92B971ECA2A3-001/control-001/cores/collection1/data
   [junit4]   2> 1479270 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_E96E92B971ECA2A3-001/control-001/cores/collection1/data/index/
   [junit4]   2> 1479270 WARN  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_E96E92B971ECA2A3-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 1479271 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_E96E92B971ECA2A3-001/control-001/cores/collection1/data/index
   [junit4]   2> 1479271 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=11, maxMergeAtOnceExplicit=40, maxMergedSegmentMB=88.4921875, floorSegmentMB=0.4912109375, forceMergeDeletesPctAllowed=5.312305620847431, segmentsPerTier=39.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
   [junit4]   2> 1479273 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(SimpleFSDirectory@/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_E96E92B971ECA2A3-001/index-SimpleFSDirectory-001 lockFactory=org.apache.lucene.store.NativeFSLockFactory@3cdea4c1),segFN=segments_1,generation=1}
   [junit4]   2> 1479273 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 1479278 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 1479278 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 1479279 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 1479279 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1479279 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1479279 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 1479279 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1479279 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1479279 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 1479280 WARN  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
   [junit4]   2> 1479283 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1479284 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1479286 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1479286 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_E96E92B971ECA2A3-001/control-001/cores/collection1/data/
   [junit4]   2> 1479287 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_E96E92B971ECA2A3-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 1479301 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1479307 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/ping,/admin/threads,/replication,/dump,/update,/admin/file,/admin/logging,/config,/update/json,/admin/system,/admin/segments,/admin/plugins,/admin/properties,/update/csv,/admin/luke,/admin/mbeans,/schema,/update/json/docs,/get,standard
   [junit4]   2> 1479309 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 1479310 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1479310 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1479311 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 1479311 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 1479312 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=15, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
   [junit4]   2> 1479314 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(SimpleFSDirectory@/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_E96E92B971ECA2A3-001/index-SimpleFSDirectory-001 lockFactory=org.apache.lucene.store.NativeFSLockFactory@3cdea4c1),segFN=segments_1,generation=1}
   [junit4]   2> 1479314 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 1479314 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@32e1b33c[collection1] main
   [junit4]   2> 1479314 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 1479320 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1479320 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 1479320 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 1479320 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 1479320 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1479321 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 1479321 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1479321 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 1479321 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1479322 INFO  (searcherExecutor-929-thread-1-processing-r:core_node1 c:control_collection n:127.0.0.1:35300_ s:shard1 x:collection1) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@32e1b33c[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1479322 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 1479322 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 1479323 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 1479323 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1517225544652947456
   [junit4]   2> 1479327 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 4.0ms to seed version buckets with highest version 1517225544652947456
   [junit4]   2> 1479327 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 1479329 INFO  (coreLoadExecutor-928-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 1479332 INFO  (coreZkRegister-922-thread-1-processing-r:core_node1 c:control_collection n:127.0.0.1:35300_ s:shard1 x:collection1) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:35300 collection:control_collection shard:shard1
   [junit4]   2> 1479333 INFO  (coreZkRegister-922-thread-1-processing-r:core_node1 c:control_collection n:127.0.0.1:35300_ s:shard1 x:collection1) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 1479342 INFO  (coreZkRegister-922-thread-1-processing-r:core_node1 c:control_collection n:127.0.0.1:35300_ s:shard1 x:collection1) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94826596390207492-core_node1-n_0000000000
   [junit4]   2> 1479343 INFO  (coreZkRegister-922-thread-1-processing-r:core_node1 c:control_collection n:127.0.0.1:35300_ s:shard1 x:collection1) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 1479347 INFO  (OverseerStateUpdate-94826596390207492-127.0.0.1:35300_-n_0000000000) [n:127.0.0.1:35300_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 1479348 INFO  (coreZkRegister-922-thread-1-processing-r:core_node1 c:control_collection n:127.0.0.1:35300_ s:shard1 x:collection1) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1479348 INFO  (coreZkRegister-922-thread-1-processing-r:core_node1 c:control_collection n:127.0.0.1:35300_ s:shard1 x:collection1) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1479348 INFO  (coreZkRegister-922-thread-1-processing-r:core_node1 c:control_collection n:127.0.0.1:35300_ s:shard1 x:collection1) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:35300/collection1/
   [junit4]   2> 1479348 INFO  (coreZkRegister-922-thread-1-processing-r:core_node1 c:control_collection n:127.0.0.1:35300_ s:shard1 x:collection1) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 1479348 INFO  (coreZkRegister-922-thread-1-processing-r:core_node1 c:control_collection n:127.0.0.1:35300_ s:shard1 x:collection1) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:35300/collection1/ has no replicas
   [junit4]   2> 1479348 INFO  (coreZkRegister-922-thread-1-processing-r:core_node1 c:control_collection n:127.0.0.1:35300_ s:shard1 x:collection1) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:35300/collection1/ shard1
   [junit4]   2> 1479351 INFO  (coreZkRegister-922-thread-1-processing-r:core_node1 c:control_collection n:127.0.0.1:35300_ s:shard1 x:collection1) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 1479356 INFO  (zkCallback-303-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1479359 INFO  (coreZkRegister-922-thread-1-processing-r:core_node1 c:control_collection n:127.0.0.1:35300_ s:shard1 x:collection1) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 1479364 INFO  (OverseerStateUpdate-94826596390207492-127.0.0.1:35300_-n_0000000000) [n:127.0.0.1:35300_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:35300",
   [junit4]   2>   "node_name":"127.0.0.1:35300_",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "core":"collection1"} current state version: 2
   [junit4]   2> 1479365 INFO  (OverseerStateUpdate-94826596390207492-127.0.0.1:35300_-n_0000000000) [n:127.0.0.1:35300_    ] o.a.s.c.o.ReplicaMutator Update state numShards=null message={
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:35300",
   [junit4]   2>   "node_name":"127.0.0.1:35300_",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "core":"collection1"}
   [junit4]   2> 1479371 INFO  (OverseerStateUpdate-94826596390207492-127.0.0.1:35300_-n_0000000000) [n:127.0.0.1:35300_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:35300",
   [junit4]   2>   "core":"collection1"} current state version: 2
   [junit4]   2> 1479474 INFO  (zkCallback-303-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1479517 INFO  (coreZkRegister-922-thread-1-processing-r:core_node1 c:control_collection n:127.0.0.1:35300_ s:shard1 x:collection1) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:35300/collection1/ and leader is http://127.0.0.1:35300/collection1/
   [junit4]   2> 1479517 INFO  (coreZkRegister-922-thread-1-processing-r:core_node1 c:control_collection n:127.0.0.1:35300_ s:shard1 x:collection1) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:35300
   [junit4]   2> 1479517 INFO  (coreZkRegister-922-thread-1-processing-r:core_node1 c:control_collection n:127.0.0.1:35300_ s:shard1 x:collection1) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 1479517 INFO  (coreZkRegister-922-thread-1-processing-r:core_node1 c:control_collection n:127.0.0.1:35300_ s:shard1 x:collection1) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 1479517 INFO  (coreZkRegister-922-thread-1-processing-r:core_node1 c:control_collection n:127.0.0.1:35300_ s:shard1 x:collection1) [n:127.0.0.1:35300_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1479520 INFO  (OverseerStateUpdate-94826596390207492-127.0.0.1:35300_-n_0000000000) [n:127.0.0.1:35300_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "base_url":"http://127.0.0.1:35300",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "node_name":"127.0.0.1:35300_",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 3
   [junit4]   2> 1479521 INFO  (OverseerStateUpdate-94826596390207492-127.0.0.1:35300_-n_0000000000) [n:127.0.0.1:35300_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "base_url":"http://127.0.0.1:35300",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "node_name":"127.0.0.1:35300_",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 1479624 INFO  (zkCallback-303-thread-1-processing-n:127.0.0.1:35300_) [n:127.0.0.1:35300_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1479813 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1479814 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1479820 INFO  (zkCallback-305-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@448af322 name:ZooKeeperConnection Watcher:127.0.0.1:36301/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1479820 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1479820 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1479820 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1479823 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 1479823 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with stateFormat=2
   [junit4]   2> 1479823 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1479824 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1479828 INFO  (zkCallback-306-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@676f09ed name:ZooKeeperConnection Watcher:127.0.0.1:36301/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1479828 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1479828 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1479831 INFO  (OverseerStateUpdate-94826596390207492-127.0.0.1:35300_-n_0000000000) [n:127.0.0.1:35300_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"create",
   [junit4]   2>   "name":"collection1",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "stateFormat":"2"} current state version: 4
   [junit4]   2> 1479832 INFO  (OverseerStateUpdate-94826596390207492-127.0.0.1:35300_-n_0000000000) [n:127.0.0.1:35300_    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 1479938 INFO  (OverseerStateUpdate-94826596390207492-127.0.0.1:35300_-n_0000000000) [n:127.0.0.1:35300_    ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/collection1/state.json
   [junit4]   2> 1480409 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_E96E92B971ECA2A3-001/shard-1-001/cores/collection1
   [junit4]   2> 1480410 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_E96E92B971ECA2A3-001/shard-1-001
   [junit4]   2> 1480416 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 1480447 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@cc2c7c7{/,null,AVAILABLE}
   [junit4]   2> 1480447 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.e.j.s.ServerConnector Started ServerConnector@47c9e05c{HTTP/1.1}{127.0.0.1:42516}
   [junit4]   2> 1480447 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.e.j.s.Server Started @1483565ms
   [junit4]   2> 1480447 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solrconfig=solrconfig.xml, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_E96E92B971ECA2A3-001/shard-1-001/cores, hostContext=/, hostPort=42516, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_E96E92B971ECA2A3-001/tempDir-001/jetty1}
   [junit4]   2> 1480448 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@3827de24
   [junit4]   2> 1480448 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_E96E92B971ECA2A3-001/shard-1-001/'
   [junit4]   2> 1480448 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1480448 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1480484 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1480485 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1480487 INFO  (zkCallback-307-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6d1dbab2 name:ZooKeeperConnection Watcher:127.0.0.1:36301/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1480488 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1480488 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1480489 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1480489 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_E96E92B971ECA2A3-001/shard-1-001/solr.xml
   [junit4]   2> 1480507 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_E96E92B971ECA2A3-001/shard-1-001/cores
   [junit4]   2> 1480508 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.CoreContainer New CoreContainer 1032188313
   [junit4]   2> 1480508 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_E96E92B971ECA2A3-001/shard-1-001/]
   [junit4]   2> 1480508 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_E96E92B971ECA2A3-001/shard-1-001/lib
   [junit4]   2> 1480508 WARN  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_E96E92B971ECA2A3-001/shard-1-001/lib).
   [junit4]   2> 1480539 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1480543 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1480544 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1480544 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1480544 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:36301/solr
   [junit4]   2> 1480544 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 1480544 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1480545 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1480559 INFO  (zkCallback-309-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@264c8074 name:ZooKeeperConnection Watcher:127.0.0.1:36301 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1480560 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[E96E92B971ECA2A3]) [    ] o.a.s.c.c.ConnectionManager Client is connected to

[...truncated too long message...]

in-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-authn-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-authz-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-changelog-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-collective-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-event-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-exception-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-journal-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-normalization-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-operational-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-referral-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-schema-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-subtree-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-interceptors-trigger-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-jdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-jdbm1-2.0.0-M2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-kerberos-codec-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-ldif-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-mavibot-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-protocol-kerberos-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-protocol-ldap-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-protocol-shared-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/apacheds-xdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/api-all-1.0.0-M20.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/bcprov-jdk15-1.45.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/cglib-nodep-2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/commons-collections-3.2.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/commons-math3-3.4.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/easymock-3.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/ehcache-core-2.4.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/hadoop-common-2.6.0-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/hadoop-hdfs-2.6.0-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/hadoop-minikdc-2.6.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jackson-annotations-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jackson-databind-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jcl-over-slf4j-1.7.7.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jersey-core-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jersey-server-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jetty-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/jetty-util-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/mina-core-2.0.0-M5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/test-lib/objenesis-1.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/analysis/icu/lucene-analyzers-icu-5.4.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/contrib/solr-analysis-extras/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/contrib/analysis-extras/lib/icu4j-54.1.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-launcher.jar:/x1/jenkins/.ant/lib/ivy-2.3.0.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-antlr.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-regexp.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-resolver.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-net.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jmf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-oro.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-xalan2.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-logging.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jsch.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit4.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-javamail.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jai.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jdepend.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bsf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-swing.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-netrexx.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bcel.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-testutil.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-log4j.jar:/x1/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.7/lib/tools.jar:/x1/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.2.0.jar com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J0-20151107_230551_577.events @/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J0-20151107_230551_577.suites -stdin
   [junit4] ERROR: JVM J0 ended with an exception: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1503)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:130)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:963)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:960)
   [junit4] 	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4] 	at java.lang.Thread.run(Thread.java:745)

BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: At least one slave process threw an exception, first: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.

Total time: 210 minutes 30 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1009 - Still Failing

[Apache Jenkins Server <je...@builds.apache.org>]

Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1009/

2 tests failed.
FAILED:  org.apache.solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR

Error Message:
Captured an uncaught exception in thread: Thread[id=10145, name=coreZkRegister-1917-thread-1, state=RUNNABLE, group=TGRP-LeaderInitiatedRecoveryOnShardRestartTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=10145, name=coreZkRegister-1917-thread-1, state=RUNNABLE, group=TGRP-LeaderInitiatedRecoveryOnShardRestartTest]
	at __randomizedtesting.SeedInfo.seed([B126B3237054D818:6FACDD5E4BF6BAEB]:0)
Caused by: java.lang.AssertionError
	at __randomizedtesting.SeedInfo.seed([B126B3237054D818]:0)
	at org.apache.solr.cloud.ZkController.updateLeaderInitiatedRecoveryState(ZkController.java:2133)
	at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:452)
	at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)
	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)
	at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:346)
	at org.apache.solr.cloud.ZkController.joinElection(ZkController.java:1113)
	at org.apache.solr.cloud.ZkController.register(ZkController.java:926)
	at org.apache.solr.cloud.ZkController.register(ZkController.java:881)
	at org.apache.solr.core.ZkContainer$2.run(ZkContainer.java:183)
	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=533, name=collection5, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=533, name=collection5, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
	at __randomizedtesting.SeedInfo.seed([B126B3237054D818:39728CF9DEA8B5E0]:0)
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:55434/_t/px: Could not find collection : awholynewstresscollection_collection5_0
	at __randomizedtesting.SeedInfo.seed([B126B3237054D818]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:895)




Build Log:
[...truncated 10131 lines...]
   [junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B126B3237054D818-001/init-core-data-001
   [junit4]   2> 0    INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[B126B3237054D818]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false)
   [junit4]   2> 26   INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[B126B3237054D818]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /_t/px
   [junit4]   2> 73   INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 76   INFO  (Thread-1) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 76   INFO  (Thread-1) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 175  INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.ZkTestServer start zk server on port:35233
   [junit4]   2> 234  INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 434  INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 498  INFO  (zkCallback-1-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@661cd479 name:ZooKeeperConnection Watcher:127.0.0.1:35233 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 499  INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 502  INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 505  INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 537  WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x150df45dd520000, likely client has closed socket
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 538  INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 544  INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 545  INFO  (zkCallback-2-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2541b7cd name:ZooKeeperConnection Watcher:127.0.0.1:35233/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 546  INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 546  INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 553  INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 563  INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 569  INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 573  INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 580  INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 595  INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 605  INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 606  INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 613  INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 614  INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 619  INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 620  INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 626  INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 628  INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 643  INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 644  INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 650  INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 652  INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 657  INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 658  INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 664  INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 664  INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 672  INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 673  INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 678  INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 679  INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 1873 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B126B3237054D818-001/control-001/cores/collection1
   [junit4]   2> 1910 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.e.j.u.log Logging initialized @5125ms
   [junit4]   2> 2083 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 2161 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@6d0b4c11{/_t/px,null,AVAILABLE}
   [junit4]   2> 2179 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.e.j.s.ServerConnector Started ServerConnector@13d10cf2{HTTP/1.1}{127.0.0.1:57881}
   [junit4]   2> 2179 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.e.j.s.Server Started @5395ms
   [junit4]   2> 2180 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/_t/px, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B126B3237054D818-001/tempDir-001/control/data, hostPort=57881, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B126B3237054D818-001/control-001/cores}
   [junit4]   2> 2189 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@3827de24
   [junit4]   2> 2235 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B126B3237054D818-001/control-001/'
   [junit4]   2> 2240 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 2240 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 2418 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2419 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2424 INFO  (zkCallback-3-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@696f551d name:ZooKeeperConnection Watcher:127.0.0.1:35233/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2424 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2425 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2430 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 2432 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B126B3237054D818-001/control-001/solr.xml
   [junit4]   2> 2585 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B126B3237054D818-001/control-001/cores
   [junit4]   2> 2645 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.CoreContainer New CoreContainer 98711131
   [junit4]   2> 2645 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B126B3237054D818-001/control-001/]
   [junit4]   2> 2645 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B126B3237054D818-001/control-001/lib
   [junit4]   2> 2646 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B126B3237054D818-001/control-001/lib).
   [junit4]   2> 2693 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 3045 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 3050 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 3052 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 3054 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:35233/solr
   [junit4]   2> 3070 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 3070 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3073 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3081 INFO  (zkCallback-5-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5ba240e5 name:ZooKeeperConnection Watcher:127.0.0.1:35233 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3082 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3082 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3095 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [n:127.0.0.1:57881__t%2Fpx    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3098 INFO  (zkCallback-6-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6320932f name:ZooKeeperConnection Watcher:127.0.0.1:35233/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3098 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [n:127.0.0.1:57881__t%2Fpx    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3110 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [n:127.0.0.1:57881__t%2Fpx    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 3132 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [n:127.0.0.1:57881__t%2Fpx    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 3142 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [n:127.0.0.1:57881__t%2Fpx    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 3152 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [n:127.0.0.1:57881__t%2Fpx    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 3159 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [n:127.0.0.1:57881__t%2Fpx    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 3175 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [n:127.0.0.1:57881__t%2Fpx    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 3182 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [n:127.0.0.1:57881__t%2Fpx    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 3187 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [n:127.0.0.1:57881__t%2Fpx    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 3191 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [n:127.0.0.1:57881__t%2Fpx    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 3193 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [n:127.0.0.1:57881__t%2Fpx    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:57881__t%2Fpx
   [junit4]   2> 3196 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [n:127.0.0.1:57881__t%2Fpx    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:57881__t%2Fpx
   [junit4]   2> 3204 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [n:127.0.0.1:57881__t%2Fpx    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 3208 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [n:127.0.0.1:57881__t%2Fpx    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 3212 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [n:127.0.0.1:57881__t%2Fpx    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 3214 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [n:127.0.0.1:57881__t%2Fpx    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94821083333656580-127.0.0.1:57881__t%2Fpx-n_0000000000
   [junit4]   2> 3223 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [n:127.0.0.1:57881__t%2Fpx    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:57881__t%2Fpx
   [junit4]   2> 3223 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [n:127.0.0.1:57881__t%2Fpx    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 3230 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [n:127.0.0.1:57881__t%2Fpx    ] o.a.s.c.Overseer Overseer (id=94821083333656580-127.0.0.1:57881__t%2Fpx-n_0000000000) starting
   [junit4]   2> 3244 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [n:127.0.0.1:57881__t%2Fpx    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 3333 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [n:127.0.0.1:57881__t%2Fpx    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 3413 INFO  (OverseerCollectionConfigSetProcessor-94821083333656580-127.0.0.1:57881__t%2Fpx-n_0000000000) [n:127.0.0.1:57881__t%2Fpx    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 3418 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [n:127.0.0.1:57881__t%2Fpx    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 3474 INFO  (OverseerStateUpdate-94821083333656580-127.0.0.1:57881__t%2Fpx-n_0000000000) [n:127.0.0.1:57881__t%2Fpx    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 3863 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [n:127.0.0.1:57881__t%2Fpx    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 3863 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [n:127.0.0.1:57881__t%2Fpx    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 3990 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [n:127.0.0.1:57881__t%2Fpx    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B126B3237054D818-001/control-001/cores
   [junit4]   2> 4071 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [n:127.0.0.1:57881__t%2Fpx    ] o.a.s.c.SolrCore Created CoreDescriptor: {config=solrconfig.xml, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B126B3237054D818-001/control-001/cores/collection1/, dataDir=data/, transient=false, shard=, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B126B3237054D818-001/control-001/cores/collection1, schema=schema.xml, configSetProperties=configsetprops.json, loadOnStartup=true, collection=control_collection, name=collection1, coreNodeName=}
   [junit4]   2> 4072 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [n:127.0.0.1:57881__t%2Fpx    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B126B3237054D818-001/control-001/cores/collection1/
   [junit4]   2> 4072 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [n:127.0.0.1:57881__t%2Fpx    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 4092 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 4092 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 4095 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [n:127.0.0.1:57881__t%2Fpx    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 4095 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [n:127.0.0.1:57881__t%2Fpx    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 4107 INFO  (OverseerStateUpdate-94821083333656580-127.0.0.1:57881__t%2Fpx-n_0000000000) [n:127.0.0.1:57881__t%2Fpx    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "roles":null,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "node_name":"127.0.0.1:57881__t%2Fpx",
   [junit4]   2>   "base_url":"http://127.0.0.1:57881/_t/px",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "numShards":"1"} current state version: 0
   [junit4]   2> 4110 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 4118 INFO  (OverseerStateUpdate-94821083333656580-127.0.0.1:57881__t%2Fpx-n_0000000000) [n:127.0.0.1:57881__t%2Fpx    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "roles":null,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "node_name":"127.0.0.1:57881__t%2Fpx",
   [junit4]   2>   "base_url":"http://127.0.0.1:57881/_t/px",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "numShards":"1"}
   [junit4]   2> 4121 INFO  (OverseerStateUpdate-94821083333656580-127.0.0.1:57881__t%2Fpx-n_0000000000) [n:127.0.0.1:57881__t%2Fpx    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 4142 INFO  (OverseerStateUpdate-94821083333656580-127.0.0.1:57881__t%2Fpx-n_0000000000) [n:127.0.0.1:57881__t%2Fpx    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 4150 INFO  (zkCallback-6-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 5115 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 5115 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 5116 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 5116 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 5118 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 5120 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B126B3237054D818-001/control-001/cores/collection1/'
   [junit4]   2> 5121 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 5121 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 5151 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 5194 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 5327 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 5570 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 5580 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 5789 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 6712 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 6805 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 6833 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 7111 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 7129 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 7136 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 7140 WARN  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 7140 WARN  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 7140 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 7146 WARN  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 7146 WARN  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 7151 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 7152 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 7153 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 7168 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B126B3237054D818-001/control-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 7172 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@6b557441
   [junit4]   2> 7247 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B126B3237054D818-001/control-001/cores/collection1/data
   [junit4]   2> 7250 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B126B3237054D818-001/control-001/cores/collection1/data/index/
   [junit4]   2> 7250 WARN  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B126B3237054D818-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 7264 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B126B3237054D818-001/control-001/cores/collection1/data/index
   [junit4]   2> 7266 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=47, maxMergeAtOnceExplicit=45, maxMergedSegmentMB=67.0009765625, floorSegmentMB=0.3671875, forceMergeDeletesPctAllowed=9.749867699448995, segmentsPerTier=25.0, maxCFSSegmentSizeMB=1.27734375, noCFSRatio=0.0
   [junit4]   2> 7332 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@31ec634b lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@166aea88),segFN=segments_1,generation=1}
   [junit4]   2> 7333 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 7346 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B126B3237054D818-001/control-001/cores/collection1/data/
   [junit4]   2> 7346 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B126B3237054D818-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 7629 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 7651 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 7656 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 7656 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 7656 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 7657 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 7679 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 7680 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 7680 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 7702 WARN  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
   [junit4]   2> 7744 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 7752 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 7755 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 7757 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 7803 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /schema,/replication,/admin/threads,/update,/admin/luke,/admin/logging,/get,/admin/system,/admin/ping,/dump,/admin/mbeans,/admin/file,/update/csv,standard,/update/json,/config,/admin/segments,/admin/plugins,/admin/properties,/update/json/docs
   [junit4]   2> 7806 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 7829 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 7829 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 7841 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 7841 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 7843 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=32, maxMergeAtOnceExplicit=24, maxMergedSegmentMB=1.921875, floorSegmentMB=1.025390625, forceMergeDeletesPctAllowed=12.156630515120758, segmentsPerTier=44.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 7849 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@31ec634b lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@166aea88),segFN=segments_1,generation=1}
   [junit4]   2> 7849 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 7870 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@25e3ff47[collection1] main
   [junit4]   2> 7881 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 7883 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 7883 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 7887 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 7887 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 7908 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 7910 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 7910 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 7910 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 7936 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 7941 INFO  (searcherExecutor-8-thread-1-processing-x:collection1 n:127.0.0.1:57881__t%2Fpx r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@25e3ff47[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 7941 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 7943 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 7943 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 7943 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1517137341494329344
   [junit4]   2> 7948 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 5.0ms to seed version buckets with highest version 1517137341494329344
   [junit4]   2> 7950 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 7952 INFO  (coreLoadExecutor-7-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 7953 INFO  (coreZkRegister-1-thread-1-processing-x:collection1 n:127.0.0.1:57881__t%2Fpx r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:57881/_t/px collection:control_collection shard:shard1
   [junit4]   2> 7961 INFO  (coreZkRegister-1-thread-1-processing-x:collection1 n:127.0.0.1:57881__t%2Fpx r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 7968 INFO  (coreZkRegister-1-thread-1-processing-x:collection1 n:127.0.0.1:57881__t%2Fpx r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94821083333656580-core_node1-n_0000000000
   [junit4]   2> 7970 INFO  (coreZkRegister-1-thread-1-processing-x:collection1 n:127.0.0.1:57881__t%2Fpx r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 7974 INFO  (coreZkRegister-1-thread-1-processing-x:collection1 n:127.0.0.1:57881__t%2Fpx r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 7974 INFO  (coreZkRegister-1-thread-1-processing-x:collection1 n:127.0.0.1:57881__t%2Fpx r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 7975 INFO  (OverseerStateUpdate-94821083333656580-127.0.0.1:57881__t%2Fpx-n_0000000000) [n:127.0.0.1:57881__t%2Fpx    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 7976 INFO  (coreZkRegister-1-thread-1-processing-x:collection1 n:127.0.0.1:57881__t%2Fpx r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:57881/_t/px/collection1/
   [junit4]   2> 7976 INFO  (coreZkRegister-1-thread-1-processing-x:collection1 n:127.0.0.1:57881__t%2Fpx r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 7977 INFO  (coreZkRegister-1-thread-1-processing-x:collection1 n:127.0.0.1:57881__t%2Fpx r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:57881/_t/px/collection1/ has no replicas
   [junit4]   2> 7977 INFO  (coreZkRegister-1-thread-1-processing-x:collection1 n:127.0.0.1:57881__t%2Fpx r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:57881/_t/px/collection1/ shard1
   [junit4]   2> 7980 INFO  (zkCallback-6-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 7982 INFO  (coreZkRegister-1-thread-1-processing-x:collection1 n:127.0.0.1:57881__t%2Fpx r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 8001 INFO  (coreZkRegister-1-thread-1-processing-x:collection1 n:127.0.0.1:57881__t%2Fpx r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 8022 INFO  (OverseerStateUpdate-94821083333656580-127.0.0.1:57881__t%2Fpx-n_0000000000) [n:127.0.0.1:57881__t%2Fpx    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:57881/_t/px",
   [junit4]   2>   "node_name":"127.0.0.1:57881__t%2Fpx",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "core":"collection1"} current state version: 2
   [junit4]   2> 8024 INFO  (OverseerStateUpdate-94821083333656580-127.0.0.1:57881__t%2Fpx-n_0000000000) [n:127.0.0.1:57881__t%2Fpx    ] o.a.s.c.o.ReplicaMutator Update state numShards=null message={
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:57881/_t/px",
   [junit4]   2>   "node_name":"127.0.0.1:57881__t%2Fpx",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "core":"collection1"}
   [junit4]   2> 8029 INFO  (OverseerStateUpdate-94821083333656580-127.0.0.1:57881__t%2Fpx-n_0000000000) [n:127.0.0.1:57881__t%2Fpx    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:57881/_t/px",
   [junit4]   2>   "core":"collection1"} current state version: 2
   [junit4]   2> 8124 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 8124 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 8127 INFO  (zkCallback-8-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@4ce7372b name:ZooKeeperConnection Watcher:127.0.0.1:35233/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 8127 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 8127 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 8128 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 8134 INFO  (zkCallback-8-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 8134 INFO  (zkCallback-6-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 8135 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 8135 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with stateFormat=2
   [junit4]   2> 8135 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 8136 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 8139 INFO  (zkCallback-9-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@dfa0111 name:ZooKeeperConnection Watcher:127.0.0.1:35233/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 8139 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 8140 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 8146 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x150df45dd520006, likely client has closed socket
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 8146 INFO  (OverseerStateUpdate-94821083333656580-127.0.0.1:57881__t%2Fpx-n_0000000000) [n:127.0.0.1:57881__t%2Fpx    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"create",
   [junit4]   2>   "name":"collection1",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "stateFormat":"2"} current state version: 3
   [junit4]   2> 8147 INFO  (OverseerStateUpdate-94821083333656580-127.0.0.1:57881__t%2Fpx-n_0000000000) [n:127.0.0.1:57881__t%2Fpx    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 8178 INFO  (coreZkRegister-1-thread-1-processing-x:collection1 n:127.0.0.1:57881__t%2Fpx r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:57881/_t/px/collection1/ and leader is http://127.0.0.1:57881/_t/px/collection1/
   [junit4]   2> 8179 INFO  (coreZkRegister-1-thread-1-processing-x:collection1 n:127.0.0.1:57881__t%2Fpx r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:57881/_t/px
   [junit4]   2> 8180 INFO  (coreZkRegister-1-thread-1-processing-x:collection1 n:127.0.0.1:57881__t%2Fpx r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 8180 INFO  (coreZkRegister-1-thread-1-processing-x:collection1 n:127.0.0.1:57881__t%2Fpx r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 8180 INFO  (coreZkRegister-1-thread-1-processing-x:collection1 n:127.0.0.1:57881__t%2Fpx r:core_node1 s:shard1 c:control_collection) [n:127.0.0.1:57881__t%2Fpx c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 8184 INFO  (OverseerStateUpdate-94821083333656580-127.0.0.1:57881__t%2Fpx-n_0000000000) [n:127.0.0.1:57881__t%2Fpx    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "roles":null,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "node_name":"127.0.0.1:57881__t%2Fpx",
   [junit4]   2>   "base_url":"http://127.0.0.1:57881/_t/px",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "numShards":"2"} current state version: 3
   [junit4]   2> 8193 INFO  (OverseerStateUpdate-94821083333656580-127.0.0.1:57881__t%2Fpx-n_0000000000) [n:127.0.0.1:57881__t%2Fpx    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "roles":null,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "node_name":"127.0.0.1:57881__t%2Fpx",
   [junit4]   2>   "base_url":"http://127.0.0.1:57881/_t/px",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "numShards":"2"}
   [junit4]   2> 8196 INFO  (OverseerStateUpdate-94821083333656580-127.0.0.1:57881__t%2Fpx-n_0000000000) [n:127.0.0.1:57881__t%2Fpx    ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/collection1/state.json
   [junit4]   2> 8306 INFO  (zkCallback-8-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 8306 INFO  (zkCallback-6-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 8493 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B126B3237054D818-001/shard-1-001/cores/collection1
   [junit4]   2> 8495 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B126B3237054D818-001/shard-1-001
   [junit4]   2> 8496 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 8502 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5504f2ca{/_t/px,null,AVAILABLE}
   [junit4]   2> 8503 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.e.j.s.ServerConnector Started ServerConnector@48b53def{HTTP/1.1}{127.0.0.1:39114}
   [junit4]   2> 8504 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.e.j.s.Server Started @11719ms
   [junit4]   2> 8504 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostPort=39114, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B126B3237054D818-001/shard-1-001/cores, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B126B3237054D818-001/tempDir-001/jetty1, solrconfig=solrconfig.xml, hostContext=/_t/px}
   [junit4]   2> 8505 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@3827de24
   [junit4]   2> 8505 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B126B3237054D818-001/shard-1-001/'
   [junit4]   2> 8506 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 8506 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 8535 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 8537 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 8538 INFO  (zkCallback-10-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@43fb37ae name:ZooKeeperConnection Watcher:127.0.0.1:35233/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 8539 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 8539 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 8542 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 8542 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B126B3237054D818-001/shard-1-001/solr.xml
   [junit4]   2> 8571 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B126B3237054D818-001/shard-1-001/cores
   [junit4]   2> 8572 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.CoreContainer New CoreContainer 1837657652
   [junit4]   2> 8572 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B126B3237054D818-001/shard-1-001/]
   [junit4]   2> 8572 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B126B3237054D818-001/shard-1-001/lib
   [junit4]   2> 8572 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B126B3237054D818-001/shard-1-001/lib).
   [junit4]   2> 8586 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 8588 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 8589 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 8590 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 8590 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:35233/solr
   [junit4]   2> 8590 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 8590 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 8606 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 8607 INFO  (zkCallback-12-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@28ad11e6 name:ZooKeeperConnection Watcher:127.0.0.1:35233 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 8607 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 8608 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 8618 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [n:127.0.0.1:39114__t%2Fpx    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 8619 INFO  (zkCallback-13-thread-1-processing-n:127.0.0.1:39114__t%2Fpx) [n:127.0.0.1:39114__t%2Fpx    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@650f5b42 name:ZooKeeperConnection Watcher:127.0.0.1:35233/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 8620 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [n:127.0.0.1:39114__t%2Fpx    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 8642 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [n:127.0.0.1:39114__t%2Fpx    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 9659 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [n:127.0.0.1:39114__t%2Fpx    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:39114__t%2Fpx
   [junit4]   2> 9662 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [n:127.0.0.1:39114__t%2Fpx    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:39114__t%2Fpx
   [junit4]   2> 9665 INFO  (zkCallback-13-thread-1-processing-n:127.0.0.1:39114__t%2Fpx) [n:127.0.0.1:39114__t%2Fpx    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 9665 INFO  (zkCallback-8-thread-1) [    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 9665 INFO  (zkCallback-6-thread-1-processing-n:127.0.0.1:57881__t%2Fpx) [n:127.0.0.1:57881__t%2Fpx    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 9670 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [n:127.0.0.1:39114__t%2Fpx    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 9672 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [n:127.0.0.1:39114__t%2Fpx    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94821083333656585-127.0.0.1:39114__t%2Fpx-n_0000000001
   [junit4]   2> 9676 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [n:127.0.0.1:39114__t%2Fpx    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/94821083333656580-127.0.0.1:57881__t%2Fpx-n_0000000000 to know if I could be the leader
   [junit4]   2> 9717 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [n:127.0.0.1:39114__t%2Fpx    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 9717 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [n:127.0.0.1:39114__t%2Fpx    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 9719 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [n:127.0.0.1:39114__t%2Fpx    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B126B3237054D818-001/shard-1-001/cores
   [junit4]   2> 9720 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [n:127.0.0.1:39114__t%2Fpx    ] o.a.s.c.SolrCore Created CoreDescriptor: {coreNodeName=, transient=false, collection=collection1, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B126B3237054D818-001/shard-1-001/cores/collection1, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B126B3237054D818-001/shard-1-001/cores/collection1/, schema=schema.xml, dataDir=data/, configSetProperties=configsetprops.json, shard=, loadOnStartup=true, name=collection1, config=solrconfig.xml}
   [junit4]   2> 9720 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [n:127.0.0.1:39114__t%2Fpx    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B126B3237054D818-001/shard-1-001/cores/collection1/
   [junit4]   2> 9721 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [n:127.0.0.1:39114__t%2Fpx    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 9722 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [n:127.0.0.1:39114__t%2Fpx    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 9723 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B126B3237054D818]) [n:127.0.0.1:39114__t%2Fpx    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 9725 INFO  (coreLoadExecutor-18-thread-1-processing-n:127.0.0.1:39114__t%2Fpx) [n:127.0.0.1:39114__t%2Fpx c:collection1   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 9725 INFO  (coreLoadExecutor-18-thread-1-processing-n:127.0.0.1:39114__t%2Fpx) [n:127.0.0.1:39114__t%2Fpx c:collection1   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 9730 INFO  (coreLoadExecutor-18-thread-1-processing-n:127.0.0.1:39114__t%2Fpx) [n:127.0.0.1:39114__t%2Fpx c:collection1   x:collection1] o.a.s.c.ZkController Registering watch for collection collection1
   [junit4]   2> 9730 INFO  (coreLoadExecutor-18-thread-1-processing-n:127.0.0.1:39114__t%2Fpx) [n:127.0.0.1:39114__t%2Fpx c:collection1   x:collection1] o.a.s.c.c.ZkStateReader addZkWatch collection1
   [junit4]   2> 9732 INFO  (OverseerStateUpdate-94821083333656580-127.0.0.1:57881__t%2Fpx-n_0000000000) [n:127.0.0.1:57881__t%2Fpx    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "roles":null,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "node_name":"127.0.0.1:39114__t%2Fpx",
   [junit4]   2>   "base_url":"http://127.0.0.1:39114/_t/px",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "numShards":"2"} current state version: 4
   [junit4]   2> 9733 INFO  (coreLoadExecutor-18-thread-1-processing-n:127.0.0.1:39114__t%2Fpx) [n:127.0.0.1:39114__t%2Fpx c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Add data for collection1 ver 0 
   [junit4]   2> 9733 INFO  (coreLoadExecutor-18-thread-1-processing-n:127.0.0.1:39114__t%2Fpx) [n:127.0.0.1:39114__t%2Fpx c:collection1   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 9733 INFO  (OverseerStateUpdate-94821083333656580-127.0.0.1:57881__t%2Fpx-n_0000000000) [n:127.0.0.1:57881__t%2Fpx    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "roles":null,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "node_name":"127.0.0.1:39114__t%2Fpx",
   [junit4]   2>   "base_url":"http://127.0.0.1:39114/_t/px",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "numShards":"2"}
   [junit4]   2> 9734 INFO  (OverseerStateUpdate-94821083333656580-127.0.0.1:57881__t%2Fpx-n_0000000000) [n:127.0.0.1:57881__t%2Fpx    ] o.a.s.c.o.ReplicaMutator Collection already exists with numShards=2
   [junit4]   2> 9734 INFO  (OverseerStateUpdate-94821083333656580-127.0.0.1:57881__t%2Fpx-n_0000000000) [n:127.0.0.1:57881__t%2Fpx    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard2
   [junit4]   2> 9841 INFO  (OverseerStateUpdate-94821083333656580-127.0.0.1:57881__t%2Fpx-n_0000000000) [n:127.0.0.1:57881__t%2Fpx    ] o.a.s.c.o.ZkStateWriter going to update_collection /collections/collection1/state.json version: 0
   [junit4]   2> 9843 INFO  (zkCallback-13-thread-1-processing-n:127.0.0.1:39114__t%2Fpx) [n:127.0.0.1:39114__t%2Fpx    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json for collection collection1 has occurred - updating... (live nodes size: 2)
   [junit4]   2> 9844 INFO  (zkCallback-13-thread-1-processing-n:127.0.0.1:39114__t%2Fpx) [n:127.0.0.1:39114__t%2Fpx    ] o.a.s.c.c.ZkStateReader Updating data for collection1 from 0 to 1 
   [junit4]   2> 10733 INFO  (coreLoadExecutor-18-thread-1-processing-n:127.0.0.1:39114__t%2Fpx) [n:127.0.0.1:39114__t%2Fpx c:collection1   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 10733 INFO  (coreLoadExecutor-18-thread-1-processing-n:127.0.0.1:39114__t%2Fpx) [n:127.0.0.1:39114__t%2Fpx c:collection1   x:collection1] o.a.s.c.ZkController Check for collection zkNode:collection1
   [junit4]   2> 10735 INFO  (coreLoadExecutor-18-thread-1-processing-n:127.0.0.1:39114__t%2Fpx) [n:127.0.0.1:39114__t%2Fpx c:collection1   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 10735 INFO  (coreLoadExecutor-18-thread-1-processing-n:127.0.0.1:39114__t%2Fpx) [n:127.0.0.1:39114__t%2Fpx c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/collection1
   [junit4]   2> 10736 INFO  (coreLoadExecutor-18-thread-1-processing-n:127.0.0.1:39114__t%2Fpx) [n:127.0.0.1:39114__t%2Fpx c:collection1   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 10736 INFO  (coreLoadExecutor-18-thread-1-processing-n:127.0.0.1:39114__t%2Fpx) [n:127.0.0.1:39114__t%2Fpx c:collection1   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B126B3237054D818-001/shard-1-001/cores/collection1/'
   [junit4]   2> 10737 INFO  (coreLoadExecutor-18-thread-1-processing-n:127.0.0.1:39114__t%2Fpx) [n:127.0.0.1:39114__t%2Fpx c:collection1   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 10737 INFO  (coreLoadExecutor-18-thread-1-processing-n:127.0.0.1:39114__t%2Fpx) [n:127.0.0.1:39114__t%2Fpx c:collection1   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 10760 INFO  (coreLoadExecutor-18-thread-1-processing-n:127.0.0.1:39114__t%2Fpx) [n:127.0.0.1:39114__t%2Fpx c:collection1   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 10787 INFO  (coreLoadExecutor-18-thread-1-processing-n:127.0.0.1:39114__t%2Fpx) [n:127.0.0.1:39114__t%2Fpx c:collection1   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 10821 INFO  (coreLoadExecutor-18-thread-1-processing-n:127.0.0.1:39114__t%2Fpx) [n:127.0.0.1:39114__t%2Fpx c:collection1   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 10873 INFO  (coreLoadExecutor-18-thread-1-processing-n:127.0.0.1:39114__t%2Fpx) [n:127.0.0.1:39114__t%2Fpx c:collection1   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 10875 INFO  (coreLoadExecutor-18-thread-1-processing-n:127.0.0.1:39114__t%2Fpx) [n:127.0.0.1:39114__t%2Fpx c:collection1   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 10980 INFO  (coreLoadExecutor-18-thread-1-processing-n:127.0.0.1:39114__t%2Fpx) [n:127.0.0.1:39114__t%2Fpx c:collection1   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 11533 INFO  (coreLoadExecutor-18-thread-1-processing-n:127.0.0.1:39114__t%2Fpx) [n:127.0.0.1:39114__t%2Fpx c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 11554 INFO  (coreLoadExecutor-18-thread-1-processing-n:127.0.0.1:39114__t%2Fpx) [n:127.0.0.1:39114__t%2Fpx c:collection1   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 11560 INFO  (coreLoadExecutor-18-thread-1-processing-n:127.0.0.

[...truncated too long message...]

Client$2.execute(SolrZkClient.java:244)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$2.execute(SolrZkClient.java:241)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:61)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.delete(SolrZkClient.java:241)
   [junit4]   2> 	at org.apache.solr.cloud.ElectionContext.cancelElection(ElectionContext.java:86)
   [junit4]   2> 	at org.apache.solr.cloud.ShardLeaderElectionContextBase.cancelElection(ElectionContext.java:169)
   [junit4]   2> 	at org.apache.solr.cloud.ShardLeaderElectionContext.cancelElection(ElectionContext.java:286)
   [junit4]   2> 	at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:468)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.access$200(LeaderElector.java:58)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector$ElectionWatcher.process(LeaderElector.java:388)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$3$1.run(SolrZkClient.java:264)
   [junit4]   2> 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
   [junit4]   2> 	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
   [junit4]   2> 	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 933410 WARN  (zkCallback-690-thread-1-processing-n:127.0.0.1:57305_) [n:127.0.0.1:57305_    ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK
   [junit4]   2> 933411 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[B126B3237054D818]) [    ] o.e.j.s.ServerConnector Stopped ServerConnector@66cee64d{HTTP/1.1}{127.0.0.1:57305}
   [junit4]   2> 933411 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[B126B3237054D818]) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@526f817c{/,null,UNAVAILABLE}
   [junit4]   2> 933418 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[B126B3237054D818]) [n:127.0.0.1:57305_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.ChaosMonkey monkey: stop shard! 40100
   [junit4]   2> 933419 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[B126B3237054D818]) [n:127.0.0.1:57305_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.ZkTestServer connecting to 127.0.0.1:34222 34222
   [junit4]   2> 933688 INFO  (Thread-6297) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:34222 34222
   [junit4]   2> 933689 WARN  (Thread-6297) [    ] o.a.s.c.ZkTestServer Watch limit violations: 
   [junit4]   2> Maximum concurrent create/delete watches above limit:
   [junit4]   2> 
   [junit4]   2> 	6	/solr/aliases.json
   [junit4]   2> 	5	/solr/security.json
   [junit4]   2> 	5	/solr/configs/conf1
   [junit4]   2> 
   [junit4]   2> Maximum concurrent data watches above limit:
   [junit4]   2> 
   [junit4]   2> 	6	/solr/clusterstate.json
   [junit4]   2> 	4	/solr/collections/collection1/state.json
   [junit4]   2> 	3	/solr/collections/all_in_lir/state.json
   [junit4]   2> 	3	/solr/overseer_elect/election/94821141467299844-127.0.0.1:40100_-n_0000000000
   [junit4]   2> 
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2> 
   [junit4]   2> 	6	/solr/collections
   [junit4]   2> 	4	/solr/live_nodes
   [junit4]   2> 	3	/solr/overseer/collection-queue-work
   [junit4]   2> 	3	/solr/overseer/queue
   [junit4]   2> 
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=LeaderInitiatedRecoveryOnShardRestartTest -Dtests.method=testRestartWithAllInLIR -Dtests.seed=B126B3237054D818 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=en -Dtests.timezone=Africa/Bissau -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
   [junit4] ERROR   46.4s J2 | LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR <<<
   [junit4]    > Throwable #1: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=10145, name=coreZkRegister-1917-thread-1, state=RUNNABLE, group=TGRP-LeaderInitiatedRecoveryOnShardRestartTest]
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([B126B3237054D818:6FACDD5E4BF6BAEB]:0)
   [junit4]    > Caused by: java.lang.AssertionError
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([B126B3237054D818]:0)
   [junit4]    > 	at org.apache.solr.cloud.ZkController.updateLeaderInitiatedRecoveryState(ZkController.java:2133)
   [junit4]    > 	at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:452)
   [junit4]    > 	at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)
   [junit4]    > 	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)
   [junit4]    > 	at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:346)
   [junit4]    > 	at org.apache.solr.cloud.ZkController.joinElection(ZkController.java:1113)
   [junit4]    > 	at org.apache.solr.cloud.ZkController.register(ZkController.java:926)
   [junit4]    > 	at org.apache.solr.cloud.ZkController.register(ZkController.java:881)
   [junit4]    > 	at org.apache.solr.core.ZkContainer$2.run(ZkContainer.java:183)
   [junit4]    > 	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)
   [junit4]    > 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   [junit4]    > 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 933692 INFO  (SUITE-LeaderInitiatedRecoveryOnShardRestartTest-seed#[B126B3237054D818]-worker) [n:127.0.0.1:57305_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: test params are: codec=CheapBastard, sim=RandomSimilarityProvider(queryNorm=false,coord=yes): {}, locale=en, timezone=Africa/Bissau
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_80 (64-bit)/cpus=4,threads=1,free=262534528,total=414187520
   [junit4]   2> NOTE: All tests run in this JVM: [TestAddFieldRealTimeGet, RequestHandlersTest, HttpPartitionTest, ChangedSchemaMergeTest, PrimitiveFieldTypeTest, TestElisionMultitermQuery, IgnoreCommitOptimizeUpdateProcessorFactoryTest, TestIndexingPerformance, SimpleFacetsTest, TestJettySolrRunner, TestFieldTypeResource, StatsComponentTest, SecurityConfHandlerTest, TestExceedMaxTermLength, TestDefaultSearchFieldResource, ZkStateWriterTest, TestCSVResponseWriter, BadIndexSchemaTest, LeaderElectionTest, TestCollationField, SmileWriterTest, RankQueryTest, SliceStateTest, AssignTest, DirectUpdateHandlerTest, TestSchemaVersionResource, HardAutoCommitTest, TestManagedSchemaFieldTypeResource, CurrencyFieldXmlFileTest, TestSmileRequest, HighlighterConfigTest, BlockCacheTest, TestShardHandlerFactory, TestUtils, RecoveryZkTest, TestCopyFieldCollectionResource, TestAnalyzeInfixSuggestions, CursorPagingTest, QueryParsingTest, HdfsSyncSliceTest, OverseerTest, PluginInfoTest, TestReload, TestSolrQueryParserDefaultOperatorResource, QueryResultKeyTest, TestBlobHandler, TestRemoveLastDynamicCopyField, DirectUpdateHandlerOptimizeTest, TestZkChroot, TermVectorComponentDistributedTest, TestLeaderInitiatedRecoveryThread, SuggesterTSTTest, CustomCollectionTest, TestSolrCloudWithKerberosAlt, FieldAnalysisRequestHandlerTest, ShowFileRequestHandlerTest, DocExpirationUpdateProcessorFactoryTest, CoreAdminCreateDiscoverTest, TestObjectReleaseTracker, TestGroupingSearch, TestSolrQueryParserResource, DocValuesMultiTest, ScriptEngineTest, TestDownShardTolerantSearch, UpdateParamsTest, TestDistribIDF, FacetPivotSmallTest, TestFuzzyAnalyzedSuggestions, SolrCoreCheckLockOnStartupTest, LeaderInitiatedRecoveryOnShardRestartTest]
   [junit4] Completed [151/545] on J2 in 46.79s, 1 test, 1 error <<< FAILURES!

[...truncated 1297 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 545 suites (7 ignored), 2158 tests, 2 errors, 32 ignored (18 assumptions) [seed: B126B3237054D818]

Total time: 189 minutes 14 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1008 - Still Failing

[Apache Jenkins Server <je...@builds.apache.org>]

Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1008/

3 tests failed.
FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Error from server at http://127.0.0.1:39301/_aki/awholynewcollection_1: Expected mime type application/octet-stream but got text/html. <html> <head> <meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/> <title>Error 500 </title> </head> <body> <h2>HTTP ERROR: 500</h2> <p>Problem accessing /_aki/awholynewcollection_1/select. Reason: <pre>    {trace=java.lang.NullPointerException  at org.apache.solr.servlet.HttpSolrCall.getCoreByCollection(HttpSolrCall.java:786)  at org.apache.solr.servlet.HttpSolrCall.init(HttpSolrCall.java:270)  at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:415)  at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:220)  at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:179)  at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)  at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:109)  at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)  at org.eclipse.jetty.servlets.UserAgentFilter.doFilter(UserAgentFilter.java:83)  at org.eclipse.jetty.servlets.GzipFilter.doFilter(GzipFilter.java:364)  at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)  at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)  at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:221)  at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)  at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)  at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)  at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)  at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)  at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)  at org.eclipse.jetty.server.Server.handle(Server.java:499)  at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)  at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)  at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)  at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)  at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)  at java.lang.Thread.run(Thread.java:745) ,code=500}</pre></p> <hr /><i><small>Powered by Jetty://</small></i> </body> </html> 

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:39301/_aki/awholynewcollection_1: Expected mime type application/octet-stream but got text/html. <html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/>
<title>Error 500 </title>
</head>
<body>
<h2>HTTP ERROR: 500</h2>
<p>Problem accessing /_aki/awholynewcollection_1/select. Reason:
<pre>    {trace=java.lang.NullPointerException
	at org.apache.solr.servlet.HttpSolrCall.getCoreByCollection(HttpSolrCall.java:786)
	at org.apache.solr.servlet.HttpSolrCall.init(HttpSolrCall.java:270)
	at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:415)
	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:220)
	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:179)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
	at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:109)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
	at org.eclipse.jetty.servlets.UserAgentFilter.doFilter(UserAgentFilter.java:83)
	at org.eclipse.jetty.servlets.GzipFilter.doFilter(GzipFilter.java:364)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:221)
	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
	at org.eclipse.jetty.server.Server.handle(Server.java:499)
	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)
	at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
	at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)
	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
	at java.lang.Thread.run(Thread.java:745)
,code=500}</pre></p>
<hr /><i><small>Powered by Jetty://</small></i>
</body>
</html>

	at __randomizedtesting.SeedInfo.seed([7AD12F3DB932BF35:F28510E717CED2CD]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:543)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:150)
	at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:943)
	at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:958)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.waitForNon403or404or503(AbstractFullDistribZkTestBase.java:1754)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testCollectionsAPI(CollectionsAPIDistributedZkTest.java:665)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test(CollectionsAPIDistributedZkTest.java:167)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1660)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:866)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:902)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:916)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:963)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:938)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:875)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:777)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:811)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:822)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.CollectionsAPIDistributedZkTest

Error Message:
5 threads leaked from SUITE scope at org.apache.solr.cloud.CollectionsAPIDistributedZkTest:     1) Thread[id=14041, name=TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]-SendThread(127.0.0.1:39606), state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)         at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:940)         at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1003)    2) Thread[id=14359, name=zkCallback-915-thread-2, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)    3) Thread[id=14043, name=zkCallback-915-thread-1, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)    4) Thread[id=14360, name=zkCallback-915-thread-3, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)    5) Thread[id=14042, name=TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]-EventThread, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:494)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 5 threads leaked from SUITE scope at org.apache.solr.cloud.CollectionsAPIDistributedZkTest: 
   1) Thread[id=14041, name=TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]-SendThread(127.0.0.1:39606), state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at java.lang.Thread.sleep(Native Method)
        at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)
        at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:940)
        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1003)
   2) Thread[id=14359, name=zkCallback-915-thread-2, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
        at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
   3) Thread[id=14043, name=zkCallback-915-thread-1, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
        at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
   4) Thread[id=14360, name=zkCallback-915-thread-3, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
        at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
   5) Thread[id=14042, name=TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]-EventThread, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:494)
	at __randomizedtesting.SeedInfo.seed([7AD12F3DB932BF35]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.CollectionsAPIDistributedZkTest

Error Message:
There are still zombie threads that couldn't be terminated:    1) Thread[id=14043, name=zkCallback-915-thread-1, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)    2) Thread[id=14359, name=zkCallback-915-thread-2, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)    3) Thread[id=14041, name=TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]-SendThread(127.0.0.1:39606), state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:994)    4) Thread[id=14360, name=zkCallback-915-thread-3, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   1) Thread[id=14043, name=zkCallback-915-thread-1, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
        at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
   2) Thread[id=14359, name=zkCallback-915-thread-2, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
        at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
   3) Thread[id=14041, name=TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]-SendThread(127.0.0.1:39606), state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at java.lang.Thread.sleep(Native Method)
        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:994)
   4) Thread[id=14360, name=zkCallback-915-thread-3, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
        at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
	at __randomizedtesting.SeedInfo.seed([7AD12F3DB932BF35]:0)




Build Log:
[...truncated 10856 lines...]
   [junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_7AD12F3DB932BF35-001/init-core-data-001
   [junit4]   2> 1432007 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[7AD12F3DB932BF35]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false)
   [junit4]   2> 1432007 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[7AD12F3DB932BF35]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /_aki/
   [junit4]   2> 1432012 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 1432012 INFO  (Thread-8885) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1432012 INFO  (Thread-8885) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 1432112 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.ZkTestServer start zk server on port:39606
   [junit4]   2> 1432112 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1432113 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1432116 INFO  (zkCallback-884-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@75cc7ce3 name:ZooKeeperConnection Watcher:127.0.0.1:39606 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1432116 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1432116 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1432116 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 1432119 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1432119 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1432121 INFO  (zkCallback-885-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@43b84db3 name:ZooKeeperConnection Watcher:127.0.0.1:39606/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1432121 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1432121 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1432121 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 1432123 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 1432124 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 1432125 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 1432127 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 1432127 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 1432129 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 1432130 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 1432131 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1432132 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1432133 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 1432133 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 1432135 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 1432135 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 1432136 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 1432137 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 1432138 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 1432138 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 1432140 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1432140 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1432141 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1432142 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1432143 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 1432143 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 1432145 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 1432145 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 1432420 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_7AD12F3DB932BF35-001/control-001/cores/collection1
   [junit4]   2> 1432422 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 1432423 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@54bcad98{/_aki,null,AVAILABLE}
   [junit4]   2> 1432424 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.e.j.s.ServerConnector Started ServerConnector@2d294442{HTTP/1.1}{127.0.0.1:39301}
   [junit4]   2> 1432424 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.e.j.s.Server Started @1435408ms
   [junit4]   2> 1432424 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_7AD12F3DB932BF35-001/control-001/cores, hostContext=/_aki, hostPort=39301, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_7AD12F3DB932BF35-001/tempDir-001/control/data}
   [junit4]   2> 1432424 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@7b3cb2c6
   [junit4]   2> 1432425 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_7AD12F3DB932BF35-001/control-001/'
   [junit4]   2> 1432425 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1432425 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1432441 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1432441 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1432443 INFO  (zkCallback-886-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2a58ee0f name:ZooKeeperConnection Watcher:127.0.0.1:39606/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1432443 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1432443 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1432444 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1432444 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_7AD12F3DB932BF35-001/control-001/solr.xml
   [junit4]   2> 1432457 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_7AD12F3DB932BF35-001/control-001/cores
   [junit4]   2> 1432457 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.CoreContainer New CoreContainer 714441687
   [junit4]   2> 1432457 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_7AD12F3DB932BF35-001/control-001/]
   [junit4]   2> 1432458 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_7AD12F3DB932BF35-001/control-001/lib
   [junit4]   2> 1432458 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_7AD12F3DB932BF35-001/control-001/lib).
   [junit4]   2> 1432465 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1432467 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1432467 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1432467 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1432467 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:39606/solr
   [junit4]   2> 1432467 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 1432468 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1432468 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1432469 INFO  (zkCallback-888-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@72a2a821 name:ZooKeeperConnection Watcher:127.0.0.1:39606 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1432470 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1432470 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1432471 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [n:127.0.0.1:39301__aki    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1432472 INFO  (zkCallback-889-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@3e055212 name:ZooKeeperConnection Watcher:127.0.0.1:39606/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1432472 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [n:127.0.0.1:39301__aki    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1432474 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [n:127.0.0.1:39301__aki    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 1432476 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [n:127.0.0.1:39301__aki    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 1432478 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [n:127.0.0.1:39301__aki    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 1432480 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [n:127.0.0.1:39301__aki    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 1432482 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [n:127.0.0.1:39301__aki    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 1432484 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [n:127.0.0.1:39301__aki    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 1432486 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [n:127.0.0.1:39301__aki    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 1432487 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [n:127.0.0.1:39301__aki    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 1432488 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [n:127.0.0.1:39301__aki    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 1432489 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [n:127.0.0.1:39301__aki    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:39301__aki
   [junit4]   2> 1432489 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [n:127.0.0.1:39301__aki    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:39301__aki
   [junit4]   2> 1432491 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [n:127.0.0.1:39301__aki    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 1432492 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [n:127.0.0.1:39301__aki    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 1432493 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [n:127.0.0.1:39301__aki    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1432494 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [n:127.0.0.1:39301__aki    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94815316352565252-127.0.0.1:39301__aki-n_0000000000
   [junit4]   2> 1432494 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [n:127.0.0.1:39301__aki    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:39301__aki
   [junit4]   2> 1432494 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [n:127.0.0.1:39301__aki    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 1432495 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [n:127.0.0.1:39301__aki    ] o.a.s.c.Overseer Overseer (id=94815316352565252-127.0.0.1:39301__aki-n_0000000000) starting
   [junit4]   2> 1432497 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [n:127.0.0.1:39301__aki    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 1432502 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [n:127.0.0.1:39301__aki    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 1432503 INFO  (OverseerCollectionConfigSetProcessor-94815316352565252-127.0.0.1:39301__aki-n_0000000000) [n:127.0.0.1:39301__aki    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 1432503 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [n:127.0.0.1:39301__aki    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1432503 INFO  (OverseerStateUpdate-94815316352565252-127.0.0.1:39301__aki-n_0000000000) [n:127.0.0.1:39301__aki    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 1432529 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [n:127.0.0.1:39301__aki    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1432529 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [n:127.0.0.1:39301__aki    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1432530 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [n:127.0.0.1:39301__aki    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_7AD12F3DB932BF35-001/control-001/cores
   [junit4]   2> 1432531 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [n:127.0.0.1:39301__aki    ] o.a.s.c.SolrCore Created CoreDescriptor: {instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_7AD12F3DB932BF35-001/control-001/cores/collection1, loadOnStartup=true, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_7AD12F3DB932BF35-001/control-001/cores/collection1/, config=solrconfig.xml, collection=control_collection, transient=false, shard=, dataDir=data/, schema=schema.xml, configSetProperties=configsetprops.json, name=collection1, coreNodeName=}
   [junit4]   2> 1432531 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [n:127.0.0.1:39301__aki    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_7AD12F3DB932BF35-001/control-001/cores/collection1/
   [junit4]   2> 1432531 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [n:127.0.0.1:39301__aki    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 1432532 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [n:127.0.0.1:39301__aki    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0
   [junit4]   2> 1432532 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [n:127.0.0.1:39301__aki    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1432532 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 1432532 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1432534 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 1432536 INFO  (OverseerStateUpdate-94815316352565252-127.0.0.1:39301__aki-n_0000000000) [n:127.0.0.1:39301__aki    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "roles":null,
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "node_name":"127.0.0.1:39301__aki",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "base_url":"http://127.0.0.1:39301/_aki"} current state version: 0
   [junit4]   2> 1432537 INFO  (OverseerStateUpdate-94815316352565252-127.0.0.1:39301__aki-n_0000000000) [n:127.0.0.1:39301__aki    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "roles":null,
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "node_name":"127.0.0.1:39301__aki",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "base_url":"http://127.0.0.1:39301/_aki"}
   [junit4]   2> 1432537 INFO  (OverseerStateUpdate-94815316352565252-127.0.0.1:39301__aki-n_0000000000) [n:127.0.0.1:39301__aki    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 1432537 INFO  (OverseerStateUpdate-94815316352565252-127.0.0.1:39301__aki-n_0000000000) [n:127.0.0.1:39301__aki    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 1432539 INFO  (zkCallback-889-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1433535 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1433535 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 1433535 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 1433535 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 1433536 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1433536 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_7AD12F3DB932BF35-001/control-001/cores/collection1/'
   [junit4]   2> 1433536 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1433536 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1433551 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 1433556 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 1433569 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 1433623 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1433624 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 1433632 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1433879 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 1433886 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 1433888 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 1433898 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1433901 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1433904 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1433905 WARN  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 1433905 WARN  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 1433905 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1433907 WARN  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 1433907 WARN  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 1433908 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0
   [junit4]   2> 1433908 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 1433908 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 1433908 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_7AD12F3DB932BF35-001/control-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 1433908 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@82b907
   [junit4]   2> 1433909 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_7AD12F3DB932BF35-001/control-001/cores/collection1/data
   [junit4]   2> 1433909 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_7AD12F3DB932BF35-001/control-001/cores/collection1/data/index/
   [junit4]   2> 1433909 WARN  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_7AD12F3DB932BF35-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 1433909 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_7AD12F3DB932BF35-001/control-001/cores/collection1/data/index
   [junit4]   2> 1433909 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=50, maxMergeAtOnceExplicit=43, maxMergedSegmentMB=59.134765625, floorSegmentMB=1.873046875, forceMergeDeletesPctAllowed=5.101132638817477, segmentsPerTier=38.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
   [junit4]   2> 1433910 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@6172c3ac lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@358c832f),segFN=segments_1,generation=1}
   [junit4]   2> 1433910 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 1433910 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_7AD12F3DB932BF35-001/control-001/cores/collection1/data/
   [junit4]   2> 1433911 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_7AD12F3DB932BF35-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 1433914 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 1433914 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 1433914 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 1433914 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1433914 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1433914 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 1433915 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1433915 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1433915 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 1433915 WARN  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
   [junit4]   2> 1433918 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1433919 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1433920 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1433921 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1433925 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /update/csv,/admin/file,/admin/properties,/admin/mbeans,/admin/ping,/admin/luke,/admin/plugins,/schema,/replication,/admin/segments,standard,/admin/logging,/dump,/update/json/docs,/update,/admin/threads,/update/json,/config,/get,/admin/system
   [junit4]   2> 1433926 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 1433927 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1433927 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1433928 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 1433928 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 1433928 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=11, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.5623707623803375]
   [junit4]   2> 1433928 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@6172c3ac lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@358c832f),segFN=segments_1,generation=1}
   [junit4]   2> 1433928 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 1433929 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@6329c12d[collection1] main
   [junit4]   2> 1433929 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 1433930 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1433930 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 1433930 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 1433930 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 1433930 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1433930 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 1433930 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1433931 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 1433931 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1433931 INFO  (searcherExecutor-3744-thread-1-processing-c:control_collection s:shard1 n:127.0.0.1:39301__aki x:collection1 r:core_node1) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@6329c12d[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1433931 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 1433931 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 1433931 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 1433931 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1517045063651164160
   [junit4]   2> 1433934 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 2.0ms to seed version buckets with highest version 1517045063651164160
   [junit4]   2> 1433934 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 1433934 INFO  (coreLoadExecutor-3743-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 1433934 INFO  (coreZkRegister-3737-thread-1-processing-c:control_collection s:shard1 n:127.0.0.1:39301__aki x:collection1 r:core_node1) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:39301/_aki collection:control_collection shard:shard1
   [junit4]   2> 1433935 INFO  (coreZkRegister-3737-thread-1-processing-c:control_collection s:shard1 n:127.0.0.1:39301__aki x:collection1 r:core_node1) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 1433940 INFO  (coreZkRegister-3737-thread-1-processing-c:control_collection s:shard1 n:127.0.0.1:39301__aki x:collection1 r:core_node1) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94815316352565252-core_node1-n_0000000000
   [junit4]   2> 1433940 INFO  (coreZkRegister-3737-thread-1-processing-c:control_collection s:shard1 n:127.0.0.1:39301__aki x:collection1 r:core_node1) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 1433942 INFO  (coreZkRegister-3737-thread-1-processing-c:control_collection s:shard1 n:127.0.0.1:39301__aki x:collection1 r:core_node1) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1433942 INFO  (coreZkRegister-3737-thread-1-processing-c:control_collection s:shard1 n:127.0.0.1:39301__aki x:collection1 r:core_node1) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1433942 INFO  (coreZkRegister-3737-thread-1-processing-c:control_collection s:shard1 n:127.0.0.1:39301__aki x:collection1 r:core_node1) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:39301/_aki/collection1/
   [junit4]   2> 1433942 INFO  (coreZkRegister-3737-thread-1-processing-c:control_collection s:shard1 n:127.0.0.1:39301__aki x:collection1 r:core_node1) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 1433942 INFO  (coreZkRegister-3737-thread-1-processing-c:control_collection s:shard1 n:127.0.0.1:39301__aki x:collection1 r:core_node1) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:39301/_aki/collection1/ has no replicas
   [junit4]   2> 1433942 INFO  (OverseerStateUpdate-94815316352565252-127.0.0.1:39301__aki-n_0000000000) [n:127.0.0.1:39301__aki    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 1433942 INFO  (coreZkRegister-3737-thread-1-processing-c:control_collection s:shard1 n:127.0.0.1:39301__aki x:collection1 r:core_node1) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:39301/_aki/collection1/ shard1
   [junit4]   2> 1433942 INFO  (coreZkRegister-3737-thread-1-processing-c:control_collection s:shard1 n:127.0.0.1:39301__aki x:collection1 r:core_node1) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 1433945 INFO  (coreZkRegister-3737-thread-1-processing-c:control_collection s:shard1 n:127.0.0.1:39301__aki x:collection1 r:core_node1) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 1433948 INFO  (OverseerStateUpdate-94815316352565252-127.0.0.1:39301__aki-n_0000000000) [n:127.0.0.1:39301__aki    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:39301/_aki",
   [junit4]   2>   "node_name":"127.0.0.1:39301__aki",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "core":"collection1"} current state version: 1
   [junit4]   2> 1433948 INFO  (OverseerStateUpdate-94815316352565252-127.0.0.1:39301__aki-n_0000000000) [n:127.0.0.1:39301__aki    ] o.a.s.c.o.ReplicaMutator Update state numShards=null message={
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:39301/_aki",
   [junit4]   2>   "node_name":"127.0.0.1:39301__aki",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "core":"collection1"}
   [junit4]   2> 1433950 INFO  (OverseerStateUpdate-94815316352565252-127.0.0.1:39301__aki-n_0000000000) [n:127.0.0.1:39301__aki    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:39301/_aki",
   [junit4]   2>   "core":"collection1"} current state version: 1
   [junit4]   2> 1434034 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1434034 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1434036 INFO  (zkCallback-891-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7f300a00 name:ZooKeeperConnection Watcher:127.0.0.1:39606/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1434036 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1434037 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1434037 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1434039 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 1434052 INFO  (zkCallback-891-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1434052 INFO  (zkCallback-889-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1434100 INFO  (coreZkRegister-3737-thread-1-processing-c:control_collection s:shard1 n:127.0.0.1:39301__aki x:collection1 r:core_node1) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:39301/_aki/collection1/ and leader is http://127.0.0.1:39301/_aki/collection1/
   [junit4]   2> 1434100 INFO  (coreZkRegister-3737-thread-1-processing-c:control_collection s:shard1 n:127.0.0.1:39301__aki x:collection1 r:core_node1) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:39301/_aki
   [junit4]   2> 1434100 INFO  (coreZkRegister-3737-thread-1-processing-c:control_collection s:shard1 n:127.0.0.1:39301__aki x:collection1 r:core_node1) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 1434100 INFO  (coreZkRegister-3737-thread-1-processing-c:control_collection s:shard1 n:127.0.0.1:39301__aki x:collection1 r:core_node1) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 1434100 INFO  (coreZkRegister-3737-thread-1-processing-c:control_collection s:shard1 n:127.0.0.1:39301__aki x:collection1 r:core_node1) [n:127.0.0.1:39301__aki c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1434102 INFO  (OverseerStateUpdate-94815316352565252-127.0.0.1:39301__aki-n_0000000000) [n:127.0.0.1:39301__aki    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "node_name":"127.0.0.1:39301__aki",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "base_url":"http://127.0.0.1:39301/_aki"} current state version: 2
   [junit4]   2> 1434103 INFO  (OverseerStateUpdate-94815316352565252-127.0.0.1:39301__aki-n_0000000000) [n:127.0.0.1:39301__aki    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "node_name":"127.0.0.1:39301__aki",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "base_url":"http://127.0.0.1:39301/_aki"}
   [junit4]   2> 1434205 INFO  (zkCallback-889-thread-1-processing-n:127.0.0.1:39301__aki) [n:127.0.0.1:39301__aki    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1434205 INFO  (zkCallback-891-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1434337 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_7AD12F3DB932BF35-001/shard-1-001/cores/collection1
   [junit4]   2> 1434337 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_7AD12F3DB932BF35-001/shard-1-001
   [junit4]   2> 1434338 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 1434340 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@769462fe{/_aki,null,AVAILABLE}
   [junit4]   2> 1434340 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.e.j.s.ServerConnector Started ServerConnector@700bc02d{HTTP/1.1}{127.0.0.1:58896}
   [junit4]   2> 1434340 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.e.j.s.Server Started @1437324ms
   [junit4]   2> 1434340 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solrconfig=solrconfig.xml, hostPort=58896, hostContext=/_aki, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_7AD12F3DB932BF35-001/shard-1-001/cores, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_7AD12F3DB932BF35-001/tempDir-001/jetty1}
   [junit4]   2> 1434341 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@7b3cb2c6
   [junit4]   2> 1434341 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_7AD12F3DB932BF35-001/shard-1-001/'
   [junit4]   2> 1434341 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1434341 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1434357 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1434357 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1434359 INFO  (zkCallback-892-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@21

[...truncated too long message...]

t test  -Dtestcase=CollectionsAPIDistributedZkTest -Dtests.seed=7AD12F3DB932BF35 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=vi -Dtests.timezone=IET -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR   0.00s J0 | CollectionsAPIDistributedZkTest (suite) <<<
   [junit4]    > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 5 threads leaked from SUITE scope at org.apache.solr.cloud.CollectionsAPIDistributedZkTest: 
   [junit4]    >    1) Thread[id=14041, name=TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]-SendThread(127.0.0.1:39606), state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    >         at java.lang.Thread.sleep(Native Method)
   [junit4]    >         at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)
   [junit4]    >         at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:940)
   [junit4]    >         at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1003)
   [junit4]    >    2) Thread[id=14359, name=zkCallback-915-thread-2, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)
   [junit4]    >         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >    3) Thread[id=14043, name=zkCallback-915-thread-1, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)
   [junit4]    >         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >    4) Thread[id=14360, name=zkCallback-915-thread-3, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)
   [junit4]    >         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >    5) Thread[id=14042, name=TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]-EventThread, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
   [junit4]    >         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]    >         at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:494)
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([7AD12F3DB932BF35]:0)Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   [junit4]    >    1) Thread[id=14043, name=zkCallback-915-thread-1, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)
   [junit4]    >         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >    2) Thread[id=14359, name=zkCallback-915-thread-2, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)
   [junit4]    >         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >    3) Thread[id=14041, name=TEST-CollectionsAPIDistributedZkTest.test-seed#[7AD12F3DB932BF35]-SendThread(127.0.0.1:39606), state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    >         at java.lang.Thread.sleep(Native Method)
   [junit4]    >         at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:994)
   [junit4]    >    4) Thread[id=14360, name=zkCallback-915-thread-3, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)
   [junit4]    >         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([7AD12F3DB932BF35]:0)
   [junit4] Completed [244/545] on J0 in 92.11s, 1 test, 3 errors <<< FAILURES!

[...truncated 921 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 545 suites (7 ignored), 2156 tests, 2 suite-level errors, 1 error, 1004 ignored (14 assumptions) [seed: 7AD12F3DB932BF35]

Total time: 165 minutes 9 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1007 - Still Failing

[Apache Jenkins Server <je...@builds.apache.org>]

Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1007/

3 tests failed.
FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=27929, name=collection4, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=27929, name=collection4, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:45597/v: collection already exists: awholynewstresscollection_collection4_1
	at __randomizedtesting.SeedInfo.seed([879120FF3D28603D]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1574)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1595)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:888)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.CollectionsAPIDistributedZkTest

Error Message:
188 threads leaked from SUITE scope at org.apache.solr.cloud.CollectionsAPIDistributedZkTest:     1) Thread[id=28412, name=searcherExecutor-3491-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)    2) Thread[id=27631, name=searcherExecutor-2994-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)    3) Thread[id=28976, name=zkCallback-722-thread-24-processing-n:127.0.0.1:44458_v-EventThread, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:494)    4) Thread[id=28419, name=searcherExecutor-3483-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)    5) Thread[id=27799, name=searcherExecutor-3120-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)    6) Thread[id=27781, name=searcherExecutor-3124-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)    7) Thread[id=27480, name=Thread-23333, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Object.wait(Native Method)         at java.lang.Object.wait(Object.java:503)         at org.apache.solr.core.CloserThread.run(CoreContainer.java:1155)    8) Thread[id=28851, name=searcherExecutor-3725-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)    9) Thread[id=28974, name=zkCallback-727-thread-5-processing-n:127.0.0.1:42668_v-EventThread, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:494)   10) Thread[id=27317, name=NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)         at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:178)         at java.lang.Thread.run(Thread.java:745)   11) Thread[id=29043, name=coreZkRegister-2881-thread-24-processing-c:awholynewstresscollection_collection0_1 x:awholynewstresscollection_collection0_1_shard4_replica1 s:shard4 r:core_node8 n:127.0.0.1:44458_v, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.cloud.ShardLeaderElectionContext.waitForReplicasToComeUp(ElectionContext.java:559)         at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:305)         at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)         at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)         at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:346)         at org.apache.solr.cloud.ZkController.joinElection(ZkController.java:1113)         at org.apache.solr.cloud.ZkController.register(ZkController.java:926)         at org.apache.solr.cloud.ZkController$RegisterCoreAsync.call(ZkController.java:233)         at java.util.concurrent.FutureTask.run(FutureTask.java:262)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   12) Thread[id=28346, name=searcherExecutor-3410-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   13) Thread[id=28984, name=coreZkRegister-2881-thread-3, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   14) Thread[id=29044, name=coreZkRegister-2881-thread-25-processing-c:awholynewstresscollection_collection0_1 x:awholynewstresscollection_collection0_1_shard4_replica1 s:shard4 r:core_node8 n:127.0.0.1:44458_v, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.cloud.ShardLeaderElectionContext.waitForReplicasToComeUp(ElectionContext.java:559)         at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:305)         at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)         at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)         at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:346)         at org.apache.solr.cloud.ZkController.joinElection(ZkController.java:1113)         at org.apache.solr.cloud.ZkController.register(ZkController.java:926)         at org.apache.solr.cloud.ZkController$RegisterCoreAsync.call(ZkController.java:233)         at java.util.concurrent.FutureTask.run(FutureTask.java:262)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   15) Thread[id=27712, name=searcherExecutor-3070-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   16) Thread[id=27624, name=searcherExecutor-2987-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   17) Thread[id=28865, name=coreZkRegister-2892-thread-5, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   18) Thread[id=27460, name=qtp438909521-27460-selector-ServerConnectorManager@135b19c5/0, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102)         at org.eclipse.jetty.io.SelectorManager$ManagedSelector.select(SelectorManager.java:600)         at org.eclipse.jetty.io.SelectorManager$ManagedSelector.run(SelectorManager.java:549)         at org.eclipse.jetty.util.thread.NonBlockingThread.run(NonBlockingThread.java:52)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)         at java.lang.Thread.run(Thread.java:745)   19) Thread[id=28374, name=searcherExecutor-3445-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   20) Thread[id=29025, name=coreZkRegister-2881-thread-11, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   21) Thread[id=29080, name=zkCallback-722-thread-34, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   22) Thread[id=28564, name=searcherExecutor-3673-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   23) Thread[id=28428, name=searcherExecutor-3485-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   24) Thread[id=28895, name=coreZkRegister-2892-thread-22, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   25) Thread[id=27801, name=searcherExecutor-3135-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   26) Thread[id=27763, name=searcherExecutor-3093-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   27) Thread[id=28263, name=qtp438909521-28263, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)         at java.lang.Thread.run(Thread.java:745)   28) Thread[id=28694, name=zkCallback-727-thread-5, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   29) Thread[id=28968, name=zkCallback-722-thread-27, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   30) Thread[id=27630, name=searcherExecutor-2991-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   31) Thread[id=28841, name=zkCallback-727-thread-13, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   32) Thread[id=28805, name=zkCallback-722-thread-17, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   33) Thread[id=28949, name=qtp105842715-28949, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)         at java.lang.Thread.run(Thread.java:745)   34) Thread[id=28893, name=searcherExecutor-3735-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   35) Thread[id=29078, name=OverseerCollectionConfigSetProcessor-94810750388404291-127.0.0.1:44458_v-n_0000000009, state=TIMED_WAITING, group=Overseer collection creation process.]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at org.apache.solr.cloud.DistributedQueue.getChildren(DistributedQueue.java:344)         at org.apache.solr.cloud.OverseerTaskQueue.peekTopN(OverseerTaskQueue.java:223)         at org.apache.solr.cloud.OverseerTaskProcessor.run(OverseerTaskProcessor.java:197)         at java.lang.Thread.run(Thread.java:745)   36) Thread[id=28211, name=qtp438909521-28211, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.cloud.ShardLeaderElectionContext.waitForReplicasToComeUp(ElectionContext.java:559)         at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:305)         at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)         at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)         at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:346)         at org.apache.solr.cloud.ZkController.joinElection(ZkController.java:1113)         at org.apache.solr.cloud.ZkController.register(ZkController.java:926)         at org.apache.solr.cloud.ZkController.register(ZkController.java:881)         at org.apache.solr.core.ZkContainer$2.run(ZkContainer.java:183)         at org.apache.solr.core.ZkContainer.registerInZk(ZkContainer.java:212)         at org.apache.solr.core.CoreContainer.registerCore(CoreContainer.java:683)         at org.apache.solr.core.CoreContainer.create(CoreContainer.java:737)         at org.apache.solr.core.CoreContainer.create(CoreContainer.java:703)         at org.apache.solr.handler.admin.CoreAdminHandler.handleCreateAction(CoreAdminHandler.java:627)         at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestInternal(CoreAdminHandler.java:212)         at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:192)         at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:151)         at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:673)         at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:441)         at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:220)         at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:179)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:109)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.eclipse.jetty.servlets.UserAgentFilter.doFilter(UserAgentFilter.java:83)         at org.eclipse.jetty.servlets.GzipFilter.doFilter(GzipFilter.java:300)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)         at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:221)         at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)         at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)         at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)         at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)         at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)         at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)         at org.eclipse.jetty.server.Server.handle(Server.java:499)         at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)         at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)         at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)         at java.lang.Thread.run(Thread.java:745)   37) Thread[id=27798, name=searcherExecutor-3127-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   38) Thread[id=27770, name=searcherExecutor-3101-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   39) Thread[id=28523, name=searcherExecutor-3638-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   40) Thread[id=27468, name=Scheduler-63946838, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1090)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   41) Thread[id=28903, name=searcherExecutor-3742-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   42) Thread[id=28967, name=zkCallback-722-thread-26-processing-n:127.0.0.1:44458_v, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.cloud.ActionThrottle.minimumWaitBetweenActions(ActionThrottle.java:82)         at org.apache.solr.update.DefaultSolrCoreState.doRecovery(DefaultSolrCoreState.java:279)         at org.apache.solr.cloud.ShardLeaderElectionContext.rejoinLeaderElection(ElectionContext.java:578)         at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:325)         at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)         at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)         at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:346)         at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:255)         at org.apache.solr.cloud.ShardLeaderElectionContext.rejoinLeaderElection(ElectionContext.java:580)         at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:325)         at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)         at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)         at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:346)         at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:255)         at org.apache.solr.cloud.ShardLeaderElectionContext.rejoinLeaderElection(ElectionContext.java:580)         at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:325)         at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)         at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)         at org.apache.solr.cloud.LeaderElector.access$200(LeaderElector.java:58)         at org.apache.solr.cloud.LeaderElector$ElectionWatcher.process(LeaderElector.java:388)         at org.apache.solr.common.cloud.SolrZkClient$3$1.run(SolrZkClient.java:264)         at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)         at java.util.concurrent.FutureTask.run(FutureTask.java:262)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   43) Thread[id=29045, name=coreZkRegister-2881-thread-26-processing-c:awholynewstresscollection_collection0_1 x:awholynewstresscollection_collection0_1_shard4_replica1 s:shard4 r:core_node8 n:127.0.0.1:44458_v, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.cloud.ShardLeaderElectionContext.waitForReplicasToComeUp(ElectionContext.java:559)         at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:305)         at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)         at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)         at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:346)         at org.apache.solr.cloud.ZkController.joinElection(ZkController.java:1113)         at org.apache.solr.cloud.ZkController.register(ZkController.java:926)         at org.apache.solr.cloud.ZkController$RegisterCoreAsync.call(ZkController.java:233)         at java.util.concurrent.FutureTask.run(FutureTask.java:262)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   44) Thread[id=28997, name=coreZkRegister-2881-thread-4-processing-c:awholynewstresscollection_collection0_1 x:awholynewstresscollection_collection0_1_shard4_replica1 s:shard4 r:core_node8 n:127.0.0.1:44458_v, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.cloud.ShardLeaderElectionContext.waitForReplicasToComeUp(ElectionContext.java:559)         at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:305)         at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)         at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)         at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:346)         at org.apache.solr.cloud.ZkController.joinElection(ZkController.java:1113)         at org.apache.solr.cloud.ZkController.register(ZkController.java:926)         at org.apache.solr.cloud.ZkController$RegisterCoreAsync.call(ZkController.java:233)         at java.util.concurrent.FutureTask.run(FutureTask.java:262)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   45) Thread[id=29074, name=zkCallback-722-thread-31-processing-n:127.0.0.1:44458_v, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.cloud.ActionThrottle.minimumWaitBetweenActions(ActionThrottle.java:82)         at org.apache.solr.update.DefaultSolrCoreState.doRecovery(DefaultSolrCoreState.java:279)         at org.apache.solr.cloud.ShardLeaderElectionContext.rejoinLeaderElection(ElectionContext.java:578)         at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:325)         at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)         at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)         at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:346)         at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:255)         at org.apache.solr.cloud.ShardLeaderElectionContext.rejoinLeaderElection(ElectionContext.java:580)         at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:325)         at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)         at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)         at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:346)         at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:255)         at org.apache.solr.cloud.ShardLeaderElectionContext.rejoinLeaderElection(ElectionContext.java:580)         at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:325)         at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)         at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)         at org.apache.solr.cloud.LeaderElector.access$200(LeaderElector.java:58)         at org.apache.solr.cloud.LeaderElector$ElectionWatcher.process(LeaderElector.java:388)         at org.apache.solr.common.cloud.SolrZkClient$3$1.run(SolrZkClient.java:264)         at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)         at java.util.concurrent.FutureTask.run(FutureTask.java:262)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   46) Thread[id=29040, name=coreZkRegister-2881-thread-21-processing-c:awholynewstresscollection_collection0_1 x:awholynewstresscollection_collection0_1_shard4_replica1 s:shard4 r:core_node8 n:127.0.0.1:44458_v, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.cloud.ShardLeaderElectionContext.waitForReplicasToComeUp(ElectionContext.java:559)         at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:305)         at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)         at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)         at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:346)         at org.apache.solr.cloud.ZkController.joinElection(ZkController.java:1113)         at org.apache.solr.cloud.ZkController.register(ZkController.java:926)         at org.apache.solr.cloud.ZkController$RegisterCoreAsync.call(ZkController.java:233)         at java.util.concurrent.FutureTask.run(FutureTask.java:262)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   47) Thread[id=28492, name=searcherExecutor-3574-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   48) Thread[id=28460, name=searcherExecutor-3547-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   49) Thread[id=28868, name=coreZkRegister-2892-thread-8, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   50) Thread[id=27489, name=Scheduler-853594404, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1079)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   51) Thread[id=28482, name=searcherExecutor-3599-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   52) Thread[id=27453, name=searcherExecutor-2887-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   53) Thread[id=29023, name=coreZkRegister-2881-thread-9, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   54) Thread[id=29029, name=coreZkRegister-2881-thread-15-processing-c:awholynewstresscollection_collection0_1 x:awholynewstresscollection_collection0_1_shard4_replica1 s:shard4 r:core_node8 n:127.0.0.1:44458_v, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.cloud.ShardLeaderElectionContext.waitForReplicasToComeUp(ElectionContext.java:559)         at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:305)         at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)         at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)         at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:346)         at org.apache.solr.cloud.ZkController.joinElection(ZkController.java:1113)         at org.apache.solr.cloud.ZkController.register(ZkController.java:926)         at org.apache.solr.cloud.ZkController$RegisterCoreAsync.call(ZkController.java:233)         at java.util.concurrent.FutureTask.run(FutureTask.java:262)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   55) Thread[id=27637, name=searcherExecutor-3010-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   56) Thread[id=28330, name=searcherExecutor-3377-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   57) Thread[id=28961, name=zkCallback-727-thread-20, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   58) Thread[id=28963, name=zkCallback-727-thread-22, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   59) Thread[id=28964, name=zkCallback-727-thread-23, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   60) Thread[id=27462, name=qtp438909521-27462-selector-ServerConnectorManager@135b19c5/1, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102)         at org.eclipse.jetty.io.SelectorManager$ManagedSelector.select(SelectorManager.java:600)         at org.eclipse.jetty.io.SelectorManager$ManagedSelector.run(SelectorManager.java:549)         at org.eclipse.jetty.util.thread.NonBlockingThread.run(NonBlockingThread.java:52)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)         at java.lang.Thread.run(Thread.java:745)   61) Thread[id=28225, name=qtp438909521-28225, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)         at java.lang.Thread.run(Thread.java:745)   62) Thread[id=28819, name=searcherExecutor-3701-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   63) Thread[id=27499, name=searcherExecutor-2903-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   64) Thread[id=28379, name=searcherExecutor-3442-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   65) Thread[id=29039, name=coreZkRegister-2881-thread-20-processing-c:awholynewstresscollection_collection0_1 x:awholynewstresscollection_collection0_1_shard4_replica1 s:shard4 r:core_node8 n:127.0.0.1:44458_v, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.cloud.ShardLeaderElectionContext.waitForReplicasToComeUp(ElectionContext.java:559)         at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:305)         at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)         at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)         at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:346)         at org.apache.solr.cloud.ZkController.joinElection(ZkController.java:1113)         at org.apache.solr.cloud.ZkController.register(ZkController.java:926)         at org.apache.solr.cloud.ZkController$RegisterCoreAsync.call(ZkController.java:233)         at java.util.concurrent.FutureTask.run(FutureTask.java:262)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   66) Thread[id=29038, name=coreZkRegister-2881-thread-19, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   67) Thread[id=28969, name=zkCallback-722-thread-28, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   68) Thread[id=29076, name=zkCallback-722-thread-33-processing-n:127.0.0.1:44458_v, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.cloud.ActionThrottle.minimumWaitBetweenActions(ActionThrottle.java:82)         at org.apache.solr.update.DefaultSolrCoreState.doRecovery(DefaultSolrCoreState.java:279)         at org.apache.solr.cloud.ShardLeaderElectionContext.rejoinLeaderElection(ElectionContext.java:578)         at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:325)         at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)         at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)         at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:346)         at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:255)         at org.apache.solr.cloud.ShardLeaderElectionContext.rejoinLeaderElection(ElectionContext.java:580)         at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:325)         at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)         at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)         at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:346)         at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:255)         at org.apache.solr.cloud.ShardLeaderElectionContext.rejoinLeaderElection(ElectionContext.java:580)         at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:325)         at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)         at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)         at org.apache.solr.cloud.LeaderElector.access$200(LeaderElector.java:58)         at org.apache.solr.cloud.LeaderElector$ElectionWatcher.process(LeaderElector.java:388)         at org.apache.solr.common.cloud.SolrZkClient$3$1.run(SolrZkClient.java:264)         at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)         at java.util.concurrent.FutureTask.run(FutureTask.java:262)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   69) Thread[id=28367, name=searcherExecutor-3433-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   70) Thread[id=27818, name=searcherExecutor-3165-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   71) Thread[id=28565, name=searcherExecutor-3662-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   72) Thread[id=27320, name=ProcessThread(sid:0 cport:-1):, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at org.apache.zookeeper.server.PrepRequestProcessor.run(PrepRequestProcessor.java:120)   73) Thread[id=27401, name=qtp1899143395-27401-selector-ServerConnectorManager@71a314d6/0, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102)         at org.eclipse.jetty.io.SelectorManager$ManagedSelector.select(SelectorManager.java:600)         at org.eclipse.jetty.io.SelectorManager$ManagedSelector.run(SelectorManager.java:549)         at org.eclipse.jetty.util.thread.NonBlockingThread.run(NonBlockingThread.java:52)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)         at java.lang.Thread.run(Thread.java:745)   74) Thread[id=28315, name=qtp1899143395-28315, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)         at java.lang.Thread.run(Thread.java:745)   75) Thread[id=28448, name=searcherExecutor-3527-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   76) Thread[id=28502, name=searcherExecutor-3607-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   77) Thread[id=28928, name=qtp105842715-28928, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)         at java.lang.Thread.run(Thread.java:745)   78) Thread[id=28975, name=zkCallback-722-thread-24-processing-n:127.0.0.1:44458_v-SendThread(127.0.0.1:37520), state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)         at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:349)         at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)   79) Thread[id=27896, name=searcherExecutor-3202-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   80) Thread[id=29072, name=zkCallback-722-thread-29-processing-n:127.0.0.1:44458_v, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.cloud.ActionThrottle.minimumWaitBetweenActions(ActionThrottle.java:82)         at org.apache.solr.update.DefaultSolrCoreState.doRecovery(DefaultSolrCoreState.java:279)         at org.apache.solr.cloud.ShardLeaderElectionContext.rejoinLeaderElection(ElectionContext.java:578)         at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:325)         at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)         at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)         at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:346)         at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:255)         at org.apache.solr.cloud.ShardLeaderElectionContext.rejoinLeaderElection(ElectionContext.java:580)         at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:325)         at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)         at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)         at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:346)         at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:255)         at org.apache.solr.cloud.ShardLeaderElectionContext.rejoinLeaderElection(ElectionContext.java:580)         at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:325)         at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)         at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)         at org.apache.solr.cloud.LeaderElector.access$200(LeaderElector.java:58)         at org.apache.solr.cloud.LeaderElector$ElectionWatcher.process(LeaderElector.java:388)         at org.apache.solr.common.cloud.SolrZkClient$3$1.run(SolrZkClient.java:264)         at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)         at java.util.concurrent.FutureTask.run(FutureTask.java:262)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   81) Thread[id=27425, name=searcherExecutor-2876-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   82) Thread[id=28896, name=coreZkRegister-2892-thread-23, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   83) Thread[id=27587, name=qtp1899143395-27587, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)         at java.lang.Thread.run(Thread.java:745)   84) Thread[id=28380, name=searcherExecutor-3449-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   85) Thread[id=29035, name=coreZkRegister-2881-thread-16-processing-c:awholynewstresscollection_collection0_1 x:awholynewstresscollection_collection0_1_shard4_replica1 s:shard4 r:core_node8 n:127.0.0.1:44458_v, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.cloud.ShardLeaderElectionContext.waitForReplicasToComeUp(ElectionContext.java:559)         at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:305)         at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)         at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)         at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:346)         at org.apache.solr.cloud.ZkController.joinElection(ZkController.java:1113)         at org.apache.solr.cloud.ZkController.register(ZkController.java:926)         at org.apache.solr.cloud.ZkController$RegisterCoreAsync.call(ZkController.java:233)         at java.util.concurrent.FutureTask.run(FutureTask.java:262)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   86) Thread[id=29036, name=coreZkRegister-2881-thread-17-processing-c:awholynewstresscollection_collection0_1 x:awholynewstresscollection_collection0_1_shard4_replica1 s:shard4 r:core_node8 n:127.0.0.1:44458_v, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.cloud.ShardLeaderElectionContext.waitForReplicasToComeUp(ElectionContext.java:559)         at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:305)         at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)         at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)         at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:346)         at org.apache.solr.cloud.ZkController.joinElection(ZkController.java:1113)         at org.apache.solr.cloud.ZkController.register(ZkController.java:926)         at org.apache.solr.cloud.ZkController$RegisterCoreAsync.call(ZkController.java:233)         at java.util.concurrent.FutureTask.run(FutureTask.java:262)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   87) Thread[id=27647, name=searcherExecutor-3028-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   88) Thread[id=28335, name=searcherExecutor-3383-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)   89) Thread[id=27429, name=qtp105842715-27429-selector-ServerConnectorManager@46ba561/0, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102)         at org.eclipse.jetty.io.SelectorManager$ManagedSelector.select(SelectorManager.java:600)         at org.eclipse.jetty.io.SelectorManager$ManagedSelector.run(SelectorManager.java:549)         at org.eclipse.jetty.util.thread.NonBlockingThread.run(NonBlockingThread.java:52)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)         at java.lang.Thread.run(Thread.java:745)   90) Thread[id=27541, name=searcherExecutor-2924-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at

[...truncated too long message...]

  >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >   153) Thread[id=28460, name=searcherExecutor-3547-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
   [junit4]    >         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >   154) Thread[id=28492, name=searcherExecutor-3574-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
   [junit4]    >         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >   155) Thread[id=29040, name=coreZkRegister-2881-thread-21, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)
   [junit4]    >         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >   156) Thread[id=29074, name=zkCallback-722-thread-31, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)
   [junit4]    >         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >   157) Thread[id=28997, name=coreZkRegister-2881-thread-4, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)
   [junit4]    >         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >   158) Thread[id=28967, name=zkCallback-722-thread-26, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)
   [junit4]    >         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >   159) Thread[id=29045, name=coreZkRegister-2881-thread-26, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)
   [junit4]    >         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >   160) Thread[id=28903, name=searcherExecutor-3742-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
   [junit4]    >         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([879120FF3D28603D]:0)
   [junit4] Completed [425/544] on J0 in 2488.36s, 1 test, 3 errors <<< FAILURES!

[...truncated 349 lines...]
   [junit4] JVM J0: stdout was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J0-20151105_040039_516.sysout
   [junit4] >>> JVM J0: stdout (verbatim) ----
   [junit4] java.lang.OutOfMemoryError: Java heap space
   [junit4] Dumping heap to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/heapdumps/java_pid28809.hprof ...
   [junit4] Heap dump file created [643691125 bytes in 6.501 secs]
   [junit4] <<< JVM J0: EOF ----

[...truncated 20 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 544 suites (7 ignored), 2153 tests, 2 suite-level errors, 1 error, 371 ignored (15 assumptions) [seed: 879120FF3D28603D]

Total time: 183 minutes 20 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1006 - Still Failing

[Apache Jenkins Server <je...@builds.apache.org>]

Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1006/

1 tests failed.
FAILED:  org.apache.lucene.search.TestGeoPointQuery.testRandomBig

Error Message:
Java heap space

Stack Trace:
java.lang.OutOfMemoryError: Java heap space
	at __randomizedtesting.SeedInfo.seed([F40CBA610A991C8F:735BC7EE9BC0600F]:0)
	at org.apache.lucene.util.ArrayUtil.grow(ArrayUtil.java:354)
	at org.apache.lucene.codecs.memory.DirectPostingsFormat$DirectField.<init>(DirectPostingsFormat.java:363)
	at org.apache.lucene.codecs.memory.DirectPostingsFormat$DirectFields.<init>(DirectPostingsFormat.java:131)
	at org.apache.lucene.codecs.memory.DirectPostingsFormat.fieldsProducer(DirectPostingsFormat.java:115)
	at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsReader.<init>(PerFieldPostingsFormat.java:261)
	at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat.fieldsProducer(PerFieldPostingsFormat.java:341)
	at org.apache.lucene.index.SegmentCoreReaders.<init>(SegmentCoreReaders.java:104)
	at org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:65)
	at org.apache.lucene.index.CheckIndex.checkIndex(CheckIndex.java:655)
	at org.apache.lucene.util.TestUtil.checkIndex(TestUtil.java:278)
	at org.apache.lucene.store.MockDirectoryWrapper.close(MockDirectoryWrapper.java:757)
	at org.apache.lucene.util.IOUtils.close(IOUtils.java:97)
	at org.apache.lucene.util.IOUtils.close(IOUtils.java:84)
	at org.apache.lucene.util.BaseGeoPointTestCase.verify(BaseGeoPointTestCase.java:766)
	at org.apache.lucene.util.BaseGeoPointTestCase.doTestRandom(BaseGeoPointTestCase.java:399)
	at org.apache.lucene.util.BaseGeoPointTestCase.testRandomBig(BaseGeoPointTestCase.java:327)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1660)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:866)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:902)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:916)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)




Build Log:
[...truncated 8503 lines...]
   [junit4] Suite: org.apache.lucene.search.TestGeoPointQuery
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestGeoPointQuery -Dtests.method=testRandomBig -Dtests.seed=F40CBA610A991C8F -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=pl_PL -Dtests.timezone=WET -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR    961s J1 | TestGeoPointQuery.testRandomBig <<<
   [junit4]    > Throwable #1: java.lang.OutOfMemoryError: Java heap space
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([F40CBA610A991C8F:735BC7EE9BC0600F]:0)
   [junit4]    > 	at org.apache.lucene.util.ArrayUtil.grow(ArrayUtil.java:354)
   [junit4]    > 	at org.apache.lucene.codecs.memory.DirectPostingsFormat$DirectField.<init>(DirectPostingsFormat.java:363)
   [junit4]    > 	at org.apache.lucene.codecs.memory.DirectPostingsFormat$DirectFields.<init>(DirectPostingsFormat.java:131)
   [junit4]    > 	at org.apache.lucene.codecs.memory.DirectPostingsFormat.fieldsProducer(DirectPostingsFormat.java:115)
   [junit4]    > 	at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsReader.<init>(PerFieldPostingsFormat.java:261)
   [junit4]    > 	at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat.fieldsProducer(PerFieldPostingsFormat.java:341)
   [junit4]    > 	at org.apache.lucene.index.SegmentCoreReaders.<init>(SegmentCoreReaders.java:104)
   [junit4]    > 	at org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:65)
   [junit4]    > 	at org.apache.lucene.index.CheckIndex.checkIndex(CheckIndex.java:655)
   [junit4]    > 	at org.apache.lucene.util.TestUtil.checkIndex(TestUtil.java:278)
   [junit4]    > 	at org.apache.lucene.store.MockDirectoryWrapper.close(MockDirectoryWrapper.java:757)
   [junit4]    > 	at org.apache.lucene.util.IOUtils.close(IOUtils.java:97)
   [junit4]    > 	at org.apache.lucene.util.IOUtils.close(IOUtils.java:84)
   [junit4]    > 	at org.apache.lucene.util.BaseGeoPointTestCase.verify(BaseGeoPointTestCase.java:766)
   [junit4]    > 	at org.apache.lucene.util.BaseGeoPointTestCase.doTestRandom(BaseGeoPointTestCase.java:399)
   [junit4]    > 	at org.apache.lucene.util.BaseGeoPointTestCase.testRandomBig(BaseGeoPointTestCase.java:327)
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/sandbox/test/J1/temp/lucene.search.TestGeoPointQuery_F40CBA610A991C8F-001
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene54): {id=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), point=PostingsFormat(name=Direct)}, docValues:{id=DocValuesFormat(name=Asserting), point=DocValuesFormat(name=Lucene54)}, sim=DefaultSimilarity, locale=pl_PL, timezone=WET
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_72 (64-bit)/cpus=4,threads=1,free=201486464,total=399507456
   [junit4]   2> NOTE: All tests run in this JVM: [TestGeoUtils, DuplicateFilterTest, TestSlowFuzzyQuery2, TestDocTermOrdsRangeFilter, TestDocValuesNumbersQuery, TestDocValuesRangeQuery, TestDocValuesTermsQuery, TestGeoPointQuery]
   [junit4] Completed [20/20] on J1 in 1050.41s, 22 tests, 1 error <<< FAILURES!

[...truncated 1 lines...]
   [junit4] JVM J1: stdout was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/sandbox/test/temp/junit4-J1-20151103_222543_473.sysout
   [junit4] >>> JVM J1: stdout (verbatim) ----
   [junit4] java.lang.OutOfMemoryError: Java heap space
   [junit4] Dumping heap to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/heapdumps/java_pid20044.hprof ...
   [junit4] Heap dump file created [544131782 bytes in 4.301 secs]
   [junit4] <<< JVM J1: EOF ----

[...truncated 10 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build.xml:471: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:2248: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/module-build.xml:58: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 20 suites, 141 tests, 1 error, 1 ignored (1 assumption) [seed: F40CBA610A991C8F]

Total time: 117 minutes 27 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1005 - Still Failing

[Apache Jenkins Server <je...@builds.apache.org>]

Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1005/

10 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.CollectionsAPIDistributedZkTest

Error Message:
Suite timeout exceeded (>= 7200000 msec).

Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
	at __randomizedtesting.SeedInfo.seed([30160A4BC582DFA1]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.CollectionsAPIDistributedZkTest

Error Message:
Captured an uncaught exception in thread: Thread[id=5614, name=collection4, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=5614, name=collection4, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:44883: collection already exists: awholynewstresscollection_collection4_0
	at __randomizedtesting.SeedInfo.seed([30160A4BC582DFA1]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1574)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1595)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:888)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.CollectionsAPIDistributedZkTest

Error Message:
Captured an uncaught exception in thread: Thread[id=5615, name=collection5, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=5615, name=collection5, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:44883: collection already exists: awholynewstresscollection_collection5_0
	at __randomizedtesting.SeedInfo.seed([30160A4BC582DFA1]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1574)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1595)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:888)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.CollectionsAPIDistributedZkTest

Error Message:
Captured an uncaught exception in thread: Thread[id=5613, name=collection3, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=5613, name=collection3, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:44883: collection already exists: awholynewstresscollection_collection3_0
	at __randomizedtesting.SeedInfo.seed([30160A4BC582DFA1]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1574)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1595)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:888)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.CollectionsAPIDistributedZkTest

Error Message:
Captured an uncaught exception in thread: Thread[id=5610, name=collection0, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=5610, name=collection0, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:44883: collection already exists: awholynewstresscollection_collection0_0
	at __randomizedtesting.SeedInfo.seed([30160A4BC582DFA1]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1574)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1595)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:888)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.CollectionsAPIDistributedZkTest

Error Message:
Captured an uncaught exception in thread: Thread[id=5612, name=collection2, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=5612, name=collection2, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: java.lang.RuntimeException: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[https://127.0.0.1:44883, https://127.0.0.1:43514, https://127.0.0.1:33172, https://127.0.0.1:43896, https://127.0.0.1:60353]
	at __randomizedtesting.SeedInfo.seed([30160A4BC582DFA1]:0)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:898)
Caused by: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[https://127.0.0.1:44883, https://127.0.0.1:43514, https://127.0.0.1:33172, https://127.0.0.1:43896, https://127.0.0.1:60353]
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:352)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:895)
Caused by: org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: https://127.0.0.1:60353
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:585)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	... 5 more
Caused by: java.net.SocketTimeoutException: Read timed out
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.read(SocketInputStream.java:152)
	at java.net.SocketInputStream.read(SocketInputStream.java:122)
	at sun.security.ssl.InputRecord.readFully(InputRecord.java:442)
	at sun.security.ssl.InputRecord.read(InputRecord.java:480)
	at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:927)
	at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1312)
	at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1339)
	at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1323)
	at org.apache.http.conn.ssl.SSLSocketFactory.connectSocket(SSLSocketFactory.java:543)
	at org.apache.http.conn.ssl.SSLSocketFactory.connectSocket(SSLSocketFactory.java:409)
	at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:177)
	at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:304)
	at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:611)
	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:446)
	at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:480)
	... 9 more


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.CollectionsAPIDistributedZkTest

Error Message:
Captured an uncaught exception in thread: Thread[id=5611, name=collection1, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=5611, name=collection1, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: java.lang.RuntimeException: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[https://127.0.0.1:44883, https://127.0.0.1:43514, https://127.0.0.1:33172, https://127.0.0.1:43896, https://127.0.0.1:60353]
	at __randomizedtesting.SeedInfo.seed([30160A4BC582DFA1]:0)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:898)
Caused by: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[https://127.0.0.1:44883, https://127.0.0.1:43514, https://127.0.0.1:33172, https://127.0.0.1:43896, https://127.0.0.1:60353]
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:352)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:895)
Caused by: org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: https://127.0.0.1:60353
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:585)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	... 5 more
Caused by: java.net.SocketTimeoutException: Read timed out
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.read(SocketInputStream.java:152)
	at java.net.SocketInputStream.read(SocketInputStream.java:122)
	at sun.security.ssl.InputRecord.readFully(InputRecord.java:442)
	at sun.security.ssl.InputRecord.read(InputRecord.java:480)
	at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:927)
	at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1312)
	at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1339)
	at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1323)
	at org.apache.http.conn.ssl.SSLSocketFactory.connectSocket(SSLSocketFactory.java:543)
	at org.apache.http.conn.ssl.SSLSocketFactory.connectSocket(SSLSocketFactory.java:409)
	at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:177)
	at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:304)
	at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:611)
	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:446)
	at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:480)
	... 9 more


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.CollectionsAPIDistributedZkTest

Error Message:
Captured an uncaught exception in thread: Thread[id=5323, name=OverseerCollectionConfigSetProcessor-94798240480362503-127.0.0.1:43514_-n_0000000000, state=RUNNABLE, group=Overseer collection creation process.]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=5323, name=OverseerCollectionConfigSetProcessor-94798240480362503-127.0.0.1:43514_-n_0000000000, state=RUNNABLE, group=Overseer collection creation process.]
Caused by: java.lang.OutOfMemoryError: Java heap space


FAILED:  org.apache.solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR

Error Message:
Captured an uncaught exception in thread: Thread[id=64661, name=coreZkRegister-5651-thread-1, state=RUNNABLE, group=TGRP-LeaderInitiatedRecoveryOnShardRestartTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=64661, name=coreZkRegister-5651-thread-1, state=RUNNABLE, group=TGRP-LeaderInitiatedRecoveryOnShardRestartTest]
Caused by: java.lang.AssertionError
	at __randomizedtesting.SeedInfo.seed([30160A4BC582DFA1]:0)
	at org.apache.solr.cloud.ZkController.updateLeaderInitiatedRecoveryState(ZkController.java:2133)
	at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:434)
	at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)
	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)
	at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:346)
	at org.apache.solr.cloud.ZkController.joinElection(ZkController.java:1113)
	at org.apache.solr.cloud.ZkController.register(ZkController.java:926)
	at org.apache.solr.cloud.ZkController.register(ZkController.java:881)
	at org.apache.solr.core.ZkContainer$2.run(ZkContainer.java:183)
	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Test abandoned because suite timeout was reached.

Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
	at __randomizedtesting.SeedInfo.seed([30160A4BC582DFA1]:0)




Build Log:
[...truncated 11313 lines...]
   [junit4] Suite: org.apache.solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_30160A4BC582DFA1-001/init-core-data-001
   [junit4]   2> 3456212 INFO  (SUITE-LeaderInitiatedRecoveryOnShardRestartTest-seed#[30160A4BC582DFA1]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /k/ls
   [junit4]   2> 3456231 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 3456243 INFO  (Thread-56474) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 3456243 INFO  (Thread-56474) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 3456343 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.ZkTestServer start zk server on port:38299
   [junit4]   2> 3456343 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3456371 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3456383 INFO  (zkCallback-1654-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7bd6f6e1 name:ZooKeeperConnection Watcher:127.0.0.1:38299 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3456383 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3456383 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3456383 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 3456385 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3456391 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3456407 INFO  (zkCallback-1655-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@682c7775 name:ZooKeeperConnection Watcher:127.0.0.1:38299/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3456407 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3456407 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3456407 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 3456409 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 3456410 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 3456411 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 3456412 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 3456412 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 3456414 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 3456414 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 3456416 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 3456416 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 3456417 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 3456417 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 3456418 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 3456419 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 3456420 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 3456420 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 3456421 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 3456421 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 3456422 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 3456423 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 3456424 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 3456424 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 3456425 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 3456425 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 3456426 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 3456427 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 3456746 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_30160A4BC582DFA1-001/control-001/cores/collection1
   [junit4]   2> 3456748 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 3456750 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@706572ee{/k/ls,null,AVAILABLE}
   [junit4]   2> 3456750 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.e.j.s.ServerConnector Started ServerConnector@4d47e9c9{HTTP/1.1}{127.0.0.1:47817}
   [junit4]   2> 3456751 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.e.j.s.Server Started @3459560ms
   [junit4]   2> 3456751 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/k/ls, hostPort=47817, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_30160A4BC582DFA1-001/control-001/cores, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_30160A4BC582DFA1-001/tempDir-001/control/data}
   [junit4]   2> 3456751 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@1fe91485
   [junit4]   2> 3456751 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_30160A4BC582DFA1-001/control-001/'
   [junit4]   2> 3456751 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 3456751 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 3456767 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3456767 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3456769 INFO  (zkCallback-1656-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@4fab9ec5 name:ZooKeeperConnection Watcher:127.0.0.1:38299/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3456769 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3456769 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3456770 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 3456770 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_30160A4BC582DFA1-001/control-001/solr.xml
   [junit4]   2> 3456785 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_30160A4BC582DFA1-001/control-001/cores
   [junit4]   2> 3456785 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.CoreContainer New CoreContainer 134198550
   [junit4]   2> 3456785 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_30160A4BC582DFA1-001/control-001/]
   [junit4]   2> 3456785 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_30160A4BC582DFA1-001/control-001/lib
   [junit4]   2> 3456785 WARN  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_30160A4BC582DFA1-001/control-001/lib).
   [junit4]   2> 3456793 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 3456795 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 3456795 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 3456795 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 3456796 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:38299/solr
   [junit4]   2> 3456796 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 3456796 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3456797 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3456799 INFO  (zkCallback-1658-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@24f37f1f name:ZooKeeperConnection Watcher:127.0.0.1:38299 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3456802 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3456803 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3456810 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [n:127.0.0.1:47817_k%2Fls    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3456819 INFO  (zkCallback-1659-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@b10c744 name:ZooKeeperConnection Watcher:127.0.0.1:38299/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3456819 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [n:127.0.0.1:47817_k%2Fls    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3456820 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [n:127.0.0.1:47817_k%2Fls    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 3456821 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [n:127.0.0.1:47817_k%2Fls    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 3456824 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [n:127.0.0.1:47817_k%2Fls    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 3456825 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [n:127.0.0.1:47817_k%2Fls    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 3456827 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [n:127.0.0.1:47817_k%2Fls    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 3456828 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [n:127.0.0.1:47817_k%2Fls    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 3456829 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [n:127.0.0.1:47817_k%2Fls    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 3456830 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [n:127.0.0.1:47817_k%2Fls    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 3456831 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [n:127.0.0.1:47817_k%2Fls    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 3456832 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [n:127.0.0.1:47817_k%2Fls    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:47817_k%2Fls
   [junit4]   2> 3456832 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [n:127.0.0.1:47817_k%2Fls    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:47817_k%2Fls
   [junit4]   2> 3456833 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [n:127.0.0.1:47817_k%2Fls    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 3456834 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [n:127.0.0.1:47817_k%2Fls    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 3456835 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [n:127.0.0.1:47817_k%2Fls    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 3456835 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [n:127.0.0.1:47817_k%2Fls    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94798410864066564-127.0.0.1:47817_k%2Fls-n_0000000000
   [junit4]   2> 3456836 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [n:127.0.0.1:47817_k%2Fls    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:47817_k%2Fls
   [junit4]   2> 3456836 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [n:127.0.0.1:47817_k%2Fls    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 3456836 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [n:127.0.0.1:47817_k%2Fls    ] o.a.s.c.Overseer Overseer (id=94798410864066564-127.0.0.1:47817_k%2Fls-n_0000000000) starting
   [junit4]   2> 3456838 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [n:127.0.0.1:47817_k%2Fls    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 3456855 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [n:127.0.0.1:47817_k%2Fls    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 3456872 INFO  (OverseerStateUpdate-94798410864066564-127.0.0.1:47817_k%2Fls-n_0000000000) [n:127.0.0.1:47817_k%2Fls    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 3456872 INFO  (OverseerStateUpdate-94798410864066564-127.0.0.1:47817_k%2Fls-n_0000000000) [n:127.0.0.1:47817_k%2Fls    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 3456882 INFO  (OverseerCollectionConfigSetProcessor-94798410864066564-127.0.0.1:47817_k%2Fls-n_0000000000) [n:127.0.0.1:47817_k%2Fls    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 3456895 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [n:127.0.0.1:47817_k%2Fls    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 3456925 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [n:127.0.0.1:47817_k%2Fls    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 3456926 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [n:127.0.0.1:47817_k%2Fls    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 3456927 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [n:127.0.0.1:47817_k%2Fls    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_30160A4BC582DFA1-001/control-001/cores
   [junit4]   2> 3456928 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [n:127.0.0.1:47817_k%2Fls    ] o.a.s.c.SolrCore Created CoreDescriptor: {config=solrconfig.xml, dataDir=data/, schema=schema.xml, coreNodeName=, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_30160A4BC582DFA1-001/control-001/cores/collection1/, name=collection1, shard=, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_30160A4BC582DFA1-001/control-001/cores/collection1, configSetProperties=configsetprops.json, collection=control_collection, transient=false, loadOnStartup=true}
   [junit4]   2> 3456929 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [n:127.0.0.1:47817_k%2Fls    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_30160A4BC582DFA1-001/control-001/cores/collection1/
   [junit4]   2> 3456929 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [n:127.0.0.1:47817_k%2Fls    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 3456943 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 3456943 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 3456944 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 3456949 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [n:127.0.0.1:47817_k%2Fls    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 3456949 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [n:127.0.0.1:47817_k%2Fls    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 3456950 INFO  (OverseerStateUpdate-94798410864066564-127.0.0.1:47817_k%2Fls-n_0000000000) [n:127.0.0.1:47817_k%2Fls    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "base_url":"http://127.0.0.1:47817/k/ls",
   [junit4]   2>   "node_name":"127.0.0.1:47817_k%2Fls",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "core":"collection1"} current state version: 0
   [junit4]   2> 3456950 INFO  (OverseerStateUpdate-94798410864066564-127.0.0.1:47817_k%2Fls-n_0000000000) [n:127.0.0.1:47817_k%2Fls    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "base_url":"http://127.0.0.1:47817/k/ls",
   [junit4]   2>   "node_name":"127.0.0.1:47817_k%2Fls",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "core":"collection1"}
   [junit4]   2> 3456950 INFO  (OverseerStateUpdate-94798410864066564-127.0.0.1:47817_k%2Fls-n_0000000000) [n:127.0.0.1:47817_k%2Fls    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 3456951 INFO  (OverseerStateUpdate-94798410864066564-127.0.0.1:47817_k%2Fls-n_0000000000) [n:127.0.0.1:47817_k%2Fls    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 3456952 INFO  (zkCallback-1659-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3456967 INFO  (zkCallback-1659-thread-2-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3457944 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 3457945 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 3457945 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 3457945 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 3457946 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 3457946 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_30160A4BC582DFA1-001/control-001/cores/collection1/'
   [junit4]   2> 3457946 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 3457946 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 3457971 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 3457979 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 3458004 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 3458104 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 3458114 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 3458121 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 3458378 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 3458385 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 3458387 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 3458396 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 3458399 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 3458402 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 3458402 WARN  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 3458403 WARN  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 3458403 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 3458403 WARN  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 3458403 WARN  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 3458404 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 3458404 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 3458405 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 3458405 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_30160A4BC582DFA1-001/control-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 3458405 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@51c9a161
   [junit4]   2> 3458405 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_30160A4BC582DFA1-001/control-001/cores/collection1/data
   [junit4]   2> 3458405 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_30160A4BC582DFA1-001/control-001/cores/collection1/data/index/
   [junit4]   2> 3458406 WARN  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_30160A4BC582DFA1-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 3458406 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_30160A4BC582DFA1-001/control-001/cores/collection1/data/index
   [junit4]   2> 3458406 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=4, maxMergeAtOnceExplicit=7, maxMergedSegmentMB=1.1650390625, floorSegmentMB=0.4404296875, forceMergeDeletesPctAllowed=4.390988511110557, segmentsPerTier=22.0, maxCFSSegmentSizeMB=1.0302734375, noCFSRatio=1.0
   [junit4]   2> 3458406 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@7e089701 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@65f2034b),segFN=segments_1,generation=1}
   [junit4]   2> 3458406 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 3458407 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_30160A4BC582DFA1-001/control-001/cores/collection1/data/
   [junit4]   2> 3458407 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_30160A4BC582DFA1-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 3458411 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 3458411 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 3458411 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 3458411 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 3458411 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 3458411 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 3458412 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 3458412 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 3458412 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 3458415 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 3458416 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 3458417 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 3458419 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 3458422 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/luke,/admin/threads,/get,/admin/mbeans,/admin/logging,/admin/segments,/replication,/update,/update/csv,standard,/update/json,/update/json/docs,/admin/ping,/admin/file,/admin/system,/admin/plugins,/config,/admin/properties,/schema
   [junit4]   2> 3458423 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 3458424 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 3458424 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 3458424 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 3458425 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 3458425 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=22, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.48114508059511085]
   [junit4]   2> 3458425 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@7e089701 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@65f2034b),segFN=segments_1,generation=1}
   [junit4]   2> 3458425 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 3458425 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@5665a2ad[collection1] main
   [junit4]   2> 3458426 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 3458426 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 3458426 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 3458427 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 3458427 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 3458427 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 3458427 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 3458427 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 3458427 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 3458427 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 3458428 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 3458428 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 3458428 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 3458428 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1516774576103620608
   [junit4]   2> 3458430 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 2.0ms to seed version buckets with highest version 1516774576103620608
   [junit4]   2> 3458430 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 3458431 INFO  (coreLoadExecutor-5613-thread-1-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 3458431 INFO  (searcherExecutor-5614-thread-1-processing-n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@5665a2ad[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 3458435 INFO  (coreZkRegister-5607-thread-1-processing-n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:47817/k/ls collection:control_collection shard:shard1
   [junit4]   2> 3458436 INFO  (coreZkRegister-5607-thread-1-processing-n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 3458439 INFO  (coreZkRegister-5607-thread-1-processing-n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94798410864066564-core_node1-n_0000000000
   [junit4]   2> 3458439 INFO  (coreZkRegister-5607-thread-1-processing-n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 3458440 INFO  (coreZkRegister-5607-thread-1-processing-n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 3458440 INFO  (coreZkRegister-5607-thread-1-processing-n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 3458441 INFO  (coreZkRegister-5607-thread-1-processing-n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:47817/k/ls/collection1/
   [junit4]   2> 3458441 INFO  (coreZkRegister-5607-thread-1-processing-n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 3458441 INFO  (coreZkRegister-5607-thread-1-processing-n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:47817/k/ls/collection1/ has no replicas
   [junit4]   2> 3458441 INFO  (OverseerStateUpdate-94798410864066564-127.0.0.1:47817_k%2Fls-n_0000000000) [n:127.0.0.1:47817_k%2Fls    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 3458441 INFO  (coreZkRegister-5607-thread-1-processing-n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:47817/k/ls/collection1/ shard1
   [junit4]   2> 3458441 INFO  (coreZkRegister-5607-thread-1-processing-n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 3458444 INFO  (coreZkRegister-5607-thread-1-processing-n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 3458447 INFO  (OverseerStateUpdate-94798410864066564-127.0.0.1:47817_k%2Fls-n_0000000000) [n:127.0.0.1:47817_k%2Fls    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:47817/k/ls",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 1
   [junit4]   2> 3458451 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3458455 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3458459 INFO  (zkCallback-1661-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7f8bbb4 name:ZooKeeperConnection Watcher:127.0.0.1:38299/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3458459 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3458459 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3458459 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 3458461 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 3458461 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with stateFormat=2
   [junit4]   2> 3458461 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3458482 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3458485 INFO  (zkCallback-1662-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7d9d9433 name:ZooKeeperConnection Watcher:127.0.0.1:38299/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3458485 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3458486 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3458488 INFO  (OverseerStateUpdate-94798410864066564-127.0.0.1:47817_k%2Fls-n_0000000000) [n:127.0.0.1:47817_k%2Fls    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"create",
   [junit4]   2>   "name":"collection1",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "stateFormat":"2"} current state version: 1
   [junit4]   2> 3458488 INFO  (OverseerStateUpdate-94798410864066564-127.0.0.1:47817_k%2Fls-n_0000000000) [n:127.0.0.1:47817_k%2Fls    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 3458488 INFO  (zkCallback-1661-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3458489 INFO  (zkCallback-1659-thread-2-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3458489 INFO  (zkCallback-1659-thread-3-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3458498 INFO  (coreZkRegister-5607-thread-1-processing-n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:47817/k/ls/collection1/ and leader is http://127.0.0.1:47817/k/ls/collection1/
   [junit4]   2> 3458498 INFO  (coreZkRegister-5607-thread-1-processing-n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:47817/k/ls
   [junit4]   2> 3458498 INFO  (coreZkRegister-5607-thread-1-processing-n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 3458498 INFO  (coreZkRegister-5607-thread-1-processing-n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 3458498 INFO  (coreZkRegister-5607-thread-1-processing-n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1) [n:127.0.0.1:47817_k%2Fls c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 3458500 INFO  (OverseerStateUpdate-94798410864066564-127.0.0.1:47817_k%2Fls-n_0000000000) [n:127.0.0.1:47817_k%2Fls    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "base_url":"http://127.0.0.1:47817/k/ls",
   [junit4]   2>   "node_name":"127.0.0.1:47817_k%2Fls",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "core":"collection1"} current state version: 2
   [junit4]   2> 3458500 INFO  (OverseerStateUpdate-94798410864066564-127.0.0.1:47817_k%2Fls-n_0000000000) [n:127.0.0.1:47817_k%2Fls    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "base_url":"http://127.0.0.1:47817/k/ls",
   [junit4]   2>   "node_name":"127.0.0.1:47817_k%2Fls",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "core":"collection1"}
   [junit4]   2> 3458501 INFO  (OverseerStateUpdate-94798410864066564-127.0.0.1:47817_k%2Fls-n_0000000000) [n:127.0.0.1:47817_k%2Fls    ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/collection1/state.json
   [junit4]   2> 3458604 INFO  (zkCallback-1661-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3458604 INFO  (zkCallback-1659-thread-3-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3458605 INFO  (zkCallback-1659-thread-2-processing-n:127.0.0.1:47817_k%2Fls) [n:127.0.0.1:47817_k%2Fls    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3458909 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_30160A4BC582DFA1-001/shard-1-001/cores/collection1
   [junit4]   2> 3458911 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_30160A4BC582DFA1-001/shard-1-001
   [junit4]   2> 3458912 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 3458967 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@6b2c8c8{/k/ls,null,AVAILABLE}
   [junit4]   2> 3458967 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.e.j.s.ServerConnector Started ServerConnector@27dde35e{HTTP/1.1}{127.0.0.1:55839}
   [junit4]   2> 3458967 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[30160A4BC582DFA1]) [    ] o.e.j.s.Server Started @3461776ms
  

[...truncated too long message...]

rverException: Timeout occured while waiting response from server at: https://127.0.0.1:60353
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:585)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
   [junit4]    > 	... 5 more
   [junit4]    > Caused by: java.net.SocketTimeoutException: Read timed out
   [junit4]    > 	at java.net.SocketInputStream.socketRead0(Native Method)
   [junit4]    > 	at java.net.SocketInputStream.read(SocketInputStream.java:152)
   [junit4]    > 	at java.net.SocketInputStream.read(SocketInputStream.java:122)
   [junit4]    > 	at sun.security.ssl.InputRecord.readFully(InputRecord.java:442)
   [junit4]    > 	at sun.security.ssl.InputRecord.read(InputRecord.java:480)
   [junit4]    > 	at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:927)
   [junit4]    > 	at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1312)
   [junit4]    > 	at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1339)
   [junit4]    > 	at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1323)
   [junit4]    > 	at org.apache.http.conn.ssl.SSLSocketFactory.connectSocket(SSLSocketFactory.java:543)
   [junit4]    > 	at org.apache.http.conn.ssl.SSLSocketFactory.connectSocket(SSLSocketFactory.java:409)
   [junit4]    > 	at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:177)
   [junit4]    > 	at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:304)
   [junit4]    > 	at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:611)
   [junit4]    > 	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:446)
   [junit4]    > 	at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
   [junit4]    > 	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
   [junit4]    > 	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
   [junit4]    > 	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:480)
   [junit4]    > 	... 9 moreThrowable #7: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=5611, name=collection1, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    > Caused by: java.lang.RuntimeException: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[https://127.0.0.1:44883, https://127.0.0.1:43514, https://127.0.0.1:33172, https://127.0.0.1:43896, https://127.0.0.1:60353]
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([30160A4BC582DFA1]:0)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:898)
   [junit4]    > Caused by: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[https://127.0.0.1:44883, https://127.0.0.1:43514, https://127.0.0.1:33172, https://127.0.0.1:43896, https://127.0.0.1:60353]
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:352)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:895)
   [junit4]    > Caused by: org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: https://127.0.0.1:60353
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:585)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
   [junit4]    > 	... 5 more
   [junit4]    > Caused by: java.net.SocketTimeoutException: Read timed out
   [junit4]    > 	at java.net.SocketInputStream.socketRead0(Native Method)
   [junit4]    > 	at java.net.SocketInputStream.read(SocketInputStream.java:152)
   [junit4]    > 	at java.net.SocketInputStream.read(SocketInputStream.java:122)
   [junit4]    > 	at sun.security.ssl.InputRecord.readFully(InputRecord.java:442)
   [junit4]    > 	at sun.security.ssl.InputRecord.read(InputRecord.java:480)
   [junit4]    > 	at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:927)
   [junit4]    > 	at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1312)
   [junit4]    > 	at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1339)
   [junit4]    > 	at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1323)
   [junit4]    > 	at org.apache.http.conn.ssl.SSLSocketFactory.connectSocket(SSLSocketFactory.java:543)
   [junit4]    > 	at org.apache.http.conn.ssl.SSLSocketFactory.connectSocket(SSLSocketFactory.java:409)
   [junit4]    > 	at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:177)
   [junit4]    > 	at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:304)
   [junit4]    > 	at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:611)
   [junit4]    > 	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:446)
   [junit4]    > 	at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
   [junit4]    > 	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
   [junit4]    > 	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
   [junit4]    > 	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:480)
   [junit4]    > 	... 9 moreThrowable #8: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=5323, name=OverseerCollectionConfigSetProcessor-94798240480362503-127.0.0.1:43514_-n_0000000000, state=RUNNABLE, group=Overseer collection creation process.]
   [junit4]    > Caused by: java.lang.OutOfMemoryError: Java heap space
   [junit4] Completed [495/543] on J1 in 12440.62s, 1 test, 9 errors <<< FAILURES!

[...truncated 142 lines...]
   [junit4] JVM J1: stdout was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J1-20151102_230455_660.sysout
   [junit4] >>> JVM J1: stdout (verbatim) ----
   [junit4] java.lang.OutOfMemoryError: Java heap space
   [junit4] Dumping heap to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/heapdumps/java_pid11049.hprof ...
   [junit4] Heap dump file created [664773063 bytes in 21.378 secs]
   [junit4] <<< JVM J1: EOF ----

   [junit4] JVM J1: stderr was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J1-20151102_230455_660.syserr
   [junit4] >>> JVM J1: stderr (verbatim) ----
   [junit4] 
   [junit4] Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "TEST-CollectionsAPIDistributedZkTest.test-seed#[30160A4BC582DFA1]-SendThread(127.0.0.1:49324)"
   [junit4] <<< JVM J1: EOF ----

[...truncated 12 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 543 suites (7 ignored), 2147 tests, 8 suite-level errors, 2 errors, 173 ignored (18 assumptions) [seed: 30160A4BC582DFA1]

Total time: 351 minutes 39 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1004 - Still Failing

[Apache Jenkins Server <je...@builds.apache.org>]

Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1004/

2 tests failed.
FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=15562, name=collection2, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=15562, name=collection2, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:50716/be_uja/jq: collection already exists: awholynewstresscollection_collection2_2
	at __randomizedtesting.SeedInfo.seed([B8CE9E547BC18E6A]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1574)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1595)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:888)


FAILED:  org.apache.solr.cloud.RollingRestartTest.test

Error Message:
Unable to restart (#3): CloudJettyRunner [url=https://127.0.0.1:39313/ynoe/p/collection1]

Stack Trace:
java.lang.AssertionError: Unable to restart (#3): CloudJettyRunner [url=https://127.0.0.1:39313/ynoe/p/collection1]
	at __randomizedtesting.SeedInfo.seed([B8CE9E547BC18E6A:309AA18ED53DE392]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.apache.solr.cloud.RollingRestartTest.restartWithRolesTest(RollingRestartTest.java:104)
	at org.apache.solr.cloud.RollingRestartTest.test(RollingRestartTest.java:53)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1660)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:866)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:902)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:916)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:963)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:938)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:875)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:777)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:811)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:822)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 10971 lines...]
   [junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B8CE9E547BC18E6A-001/init-core-data-001
   [junit4]   2> 1547028 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[B8CE9E547BC18E6A]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true)
   [junit4]   2> 1547028 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[B8CE9E547BC18E6A]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /be_uja/jq
   [junit4]   2> 1547032 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 1547033 INFO  (Thread-10104) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1547033 INFO  (Thread-10104) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 1547133 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.ZkTestServer start zk server on port:39006
   [junit4]   2> 1547133 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1547133 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1547138 INFO  (zkCallback-960-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@27fc8de3 name:ZooKeeperConnection Watcher:127.0.0.1:39006 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1547138 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1547138 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1547138 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 1547141 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1547142 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1547143 INFO  (zkCallback-961-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@62d68c34 name:ZooKeeperConnection Watcher:127.0.0.1:39006/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1547143 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1547143 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1547143 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 1547146 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 1547148 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 1547150 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 1547152 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 1547152 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 1547155 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 1547155 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 1547157 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1547158 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1547159 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 1547160 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 1547161 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 1547162 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 1547163 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 1547164 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 1547166 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 1547166 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 1547168 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1547168 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1547170 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1547170 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1547172 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 1547172 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 1547174 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 1547174 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 1547446 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B8CE9E547BC18E6A-001/control-001/cores/collection1
   [junit4]   2> 1547447 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 1547449 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5852e84a{/be_uja/jq,null,AVAILABLE}
   [junit4]   2> 1547452 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.e.j.s.ServerConnector Started ServerConnector@4827a1fb{HTTP/1.1}{127.0.0.1:48000}
   [junit4]   2> 1547452 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.e.j.s.Server Started @1550123ms
   [junit4]   2> 1547452 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B8CE9E547BC18E6A-001/control-001/cores, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B8CE9E547BC18E6A-001/tempDir-001/control/data, hostContext=/be_uja/jq, hostPort=48000}
   [junit4]   2> 1547452 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@1fe91485
   [junit4]   2> 1547452 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B8CE9E547BC18E6A-001/control-001/'
   [junit4]   2> 1547452 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1547452 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1547468 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1547469 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1547470 INFO  (zkCallback-962-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@4b6043c7 name:ZooKeeperConnection Watcher:127.0.0.1:39006/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1547470 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1547471 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1547471 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1547471 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B8CE9E547BC18E6A-001/control-001/solr.xml
   [junit4]   2> 1547488 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B8CE9E547BC18E6A-001/control-001/cores
   [junit4]   2> 1547488 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.CoreContainer New CoreContainer 1617162446
   [junit4]   2> 1547488 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B8CE9E547BC18E6A-001/control-001/]
   [junit4]   2> 1547488 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B8CE9E547BC18E6A-001/control-001/lib
   [junit4]   2> 1547488 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B8CE9E547BC18E6A-001/control-001/lib).
   [junit4]   2> 1547496 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1547497 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1547498 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1547498 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1547498 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:39006/solr
   [junit4]   2> 1547498 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 1547498 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1547499 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1547500 INFO  (zkCallback-964-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@593949a0 name:ZooKeeperConnection Watcher:127.0.0.1:39006 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1547501 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1547501 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1547502 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [n:127.0.0.1:48000_be_uja%2Fjq    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1547504 INFO  (zkCallback-965-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@59248b28 name:ZooKeeperConnection Watcher:127.0.0.1:39006/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1547504 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [n:127.0.0.1:48000_be_uja%2Fjq    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1547505 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [n:127.0.0.1:48000_be_uja%2Fjq    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 1547507 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [n:127.0.0.1:48000_be_uja%2Fjq    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 1547510 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [n:127.0.0.1:48000_be_uja%2Fjq    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 1547512 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [n:127.0.0.1:48000_be_uja%2Fjq    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 1547515 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [n:127.0.0.1:48000_be_uja%2Fjq    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 1547517 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [n:127.0.0.1:48000_be_uja%2Fjq    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 1547518 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [n:127.0.0.1:48000_be_uja%2Fjq    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 1547520 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [n:127.0.0.1:48000_be_uja%2Fjq    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 1547521 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [n:127.0.0.1:48000_be_uja%2Fjq    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 1547522 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [n:127.0.0.1:48000_be_uja%2Fjq    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:48000_be_uja%2Fjq
   [junit4]   2> 1547522 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [n:127.0.0.1:48000_be_uja%2Fjq    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:48000_be_uja%2Fjq
   [junit4]   2> 1547524 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [n:127.0.0.1:48000_be_uja%2Fjq    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 1547525 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [n:127.0.0.1:48000_be_uja%2Fjq    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 1547527 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [n:127.0.0.1:48000_be_uja%2Fjq    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1547527 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [n:127.0.0.1:48000_be_uja%2Fjq    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94792819541475332-127.0.0.1:48000_be_uja%2Fjq-n_0000000000
   [junit4]   2> 1547528 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [n:127.0.0.1:48000_be_uja%2Fjq    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:48000_be_uja%2Fjq
   [junit4]   2> 1547528 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [n:127.0.0.1:48000_be_uja%2Fjq    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 1547529 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [n:127.0.0.1:48000_be_uja%2Fjq    ] o.a.s.c.Overseer Overseer (id=94792819541475332-127.0.0.1:48000_be_uja%2Fjq-n_0000000000) starting
   [junit4]   2> 1547531 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [n:127.0.0.1:48000_be_uja%2Fjq    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 1547538 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [n:127.0.0.1:48000_be_uja%2Fjq    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 1547538 INFO  (OverseerCollectionConfigSetProcessor-94792819541475332-127.0.0.1:48000_be_uja%2Fjq-n_0000000000) [n:127.0.0.1:48000_be_uja%2Fjq    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 1547538 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [n:127.0.0.1:48000_be_uja%2Fjq    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1547539 INFO  (OverseerStateUpdate-94792819541475332-127.0.0.1:48000_be_uja%2Fjq-n_0000000000) [n:127.0.0.1:48000_be_uja%2Fjq    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 1547561 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [n:127.0.0.1:48000_be_uja%2Fjq    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1547561 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [n:127.0.0.1:48000_be_uja%2Fjq    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1547563 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [n:127.0.0.1:48000_be_uja%2Fjq    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B8CE9E547BC18E6A-001/control-001/cores
   [junit4]   2> 1547563 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [n:127.0.0.1:48000_be_uja%2Fjq    ] o.a.s.c.SolrCore Created CoreDescriptor: {config=solrconfig.xml, collection=control_collection, dataDir=data/, loadOnStartup=true, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B8CE9E547BC18E6A-001/control-001/cores/collection1/, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B8CE9E547BC18E6A-001/control-001/cores/collection1, schema=schema.xml, coreNodeName=, shard=, name=collection1, configSetProperties=configsetprops.json, transient=false}
   [junit4]   2> 1547563 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [n:127.0.0.1:48000_be_uja%2Fjq    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B8CE9E547BC18E6A-001/control-001/cores/collection1/
   [junit4]   2> 1547563 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [n:127.0.0.1:48000_be_uja%2Fjq    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 1547564 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [n:127.0.0.1:48000_be_uja%2Fjq    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 1547564 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [n:127.0.0.1:48000_be_uja%2Fjq    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1547565 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 1547565 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1547566 INFO  (OverseerStateUpdate-94792819541475332-127.0.0.1:48000_be_uja%2Fjq-n_0000000000) [n:127.0.0.1:48000_be_uja%2Fjq    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "roles":null,
   [junit4]   2>   "node_name":"127.0.0.1:48000_be_uja%2Fjq",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "base_url":"http://127.0.0.1:48000/be_uja/jq",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "state":"down"} current state version: 0
   [junit4]   2> 1547567 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 1547567 INFO  (OverseerStateUpdate-94792819541475332-127.0.0.1:48000_be_uja%2Fjq-n_0000000000) [n:127.0.0.1:48000_be_uja%2Fjq    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "roles":null,
   [junit4]   2>   "node_name":"127.0.0.1:48000_be_uja%2Fjq",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "base_url":"http://127.0.0.1:48000/be_uja/jq",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "state":"down"}
   [junit4]   2> 1547567 INFO  (OverseerStateUpdate-94792819541475332-127.0.0.1:48000_be_uja%2Fjq-n_0000000000) [n:127.0.0.1:48000_be_uja%2Fjq    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 1547568 INFO  (OverseerStateUpdate-94792819541475332-127.0.0.1:48000_be_uja%2Fjq-n_0000000000) [n:127.0.0.1:48000_be_uja%2Fjq    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 1547569 INFO  (zkCallback-965-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1548567 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1548567 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 1548568 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 1548568 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 1548569 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1548569 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B8CE9E547BC18E6A-001/control-001/cores/collection1/'
   [junit4]   2> 1548569 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1548569 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1548611 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 1548617 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 1548631 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 1548659 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1548660 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 1548667 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1548911 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 1548918 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 1548920 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 1548928 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1548931 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1548934 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1548934 WARN  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 1548935 WARN  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 1548935 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1548935 WARN  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 1548935 WARN  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 1548936 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 1548936 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 1548936 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 1548936 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B8CE9E547BC18E6A-001/control-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 1548937 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@459304b2
   [junit4]   2> 1548937 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B8CE9E547BC18E6A-001/control-001/cores/collection1/data
   [junit4]   2> 1548938 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B8CE9E547BC18E6A-001/control-001/cores/collection1/data/index/
   [junit4]   2> 1548938 WARN  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B8CE9E547BC18E6A-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 1548938 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B8CE9E547BC18E6A-001/control-001/cores/collection1/data/index
   [junit4]   2> 1548938 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=6, maxMergeAtOnceExplicit=7, maxMergedSegmentMB=27.544921875, floorSegmentMB=0.615234375, forceMergeDeletesPctAllowed=25.515157929851853, segmentsPerTier=43.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.8379542938702577
   [junit4]   2> 1548939 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=RawDirectoryWrapper(RAMDirectory@dc20826 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@3da4fdd8),segFN=segments_1,generation=1}
   [junit4]   2> 1548939 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 1548939 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B8CE9E547BC18E6A-001/control-001/cores/collection1/data/
   [junit4]   2> 1548939 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B8CE9E547BC18E6A-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 1548943 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 1548944 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 1548944 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 1548944 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1548944 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1548944 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 1548944 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1548944 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1548944 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 1548947 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1548948 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1548950 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1548951 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1548954 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/plugins,/update/json/docs,/admin/system,/schema,/update/json,/admin/threads,/admin/file,/admin/mbeans,/admin/logging,/get,/admin/ping,standard,/admin/properties,/admin/segments,/config,/replication,/update/csv,/update,/admin/luke
   [junit4]   2> 1548955 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 1548956 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1548956 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1548957 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 1548957 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 1548957 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@dfcee1d
   [junit4]   2> 1548957 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=RawDirectoryWrapper(RAMDirectory@dc20826 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@3da4fdd8),segFN=segments_1,generation=1}
   [junit4]   2> 1548958 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 1548958 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@5723c25f[collection1] main
   [junit4]   2> 1548958 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 1548959 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1548959 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 1548959 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 1548959 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 1548959 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1548959 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 1548960 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1548960 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 1548960 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1548960 INFO  (searcherExecutor-3204-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq r:core_node1 x:collection1 c:control_collection s:shard1) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@5723c25f[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1548960 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 1548961 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 1548961 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 1548961 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1516685114684211200
   [junit4]   2> 1548963 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 2.0ms to seed version buckets with highest version 1516685114684211200
   [junit4]   2> 1548963 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 1548963 INFO  (coreLoadExecutor-3203-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 1548964 INFO  (coreZkRegister-3197-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq r:core_node1 x:collection1 c:control_collection s:shard1) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:48000/be_uja/jq collection:control_collection shard:shard1
   [junit4]   2> 1548964 INFO  (coreZkRegister-3197-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq r:core_node1 x:collection1 c:control_collection s:shard1) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 1548968 INFO  (coreZkRegister-3197-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq r:core_node1 x:collection1 c:control_collection s:shard1) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94792819541475332-core_node1-n_0000000000
   [junit4]   2> 1548968 INFO  (coreZkRegister-3197-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq r:core_node1 x:collection1 c:control_collection s:shard1) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 1548970 INFO  (coreZkRegister-3197-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq r:core_node1 x:collection1 c:control_collection s:shard1) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1548970 INFO  (coreZkRegister-3197-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq r:core_node1 x:collection1 c:control_collection s:shard1) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1548970 INFO  (coreZkRegister-3197-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq r:core_node1 x:collection1 c:control_collection s:shard1) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:48000/be_uja/jq/collection1/
   [junit4]   2> 1548970 INFO  (coreZkRegister-3197-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq r:core_node1 x:collection1 c:control_collection s:shard1) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 1548970 INFO  (coreZkRegister-3197-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq r:core_node1 x:collection1 c:control_collection s:shard1) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:48000/be_uja/jq/collection1/ has no replicas
   [junit4]   2> 1548970 INFO  (coreZkRegister-3197-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq r:core_node1 x:collection1 c:control_collection s:shard1) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:48000/be_uja/jq/collection1/ shard1
   [junit4]   2> 1548970 INFO  (OverseerStateUpdate-94792819541475332-127.0.0.1:48000_be_uja%2Fjq-n_0000000000) [n:127.0.0.1:48000_be_uja%2Fjq    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 1548970 INFO  (coreZkRegister-3197-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq r:core_node1 x:collection1 c:control_collection s:shard1) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 1548973 INFO  (coreZkRegister-3197-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq r:core_node1 x:collection1 c:control_collection s:shard1) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 1548975 INFO  (OverseerStateUpdate-94792819541475332-127.0.0.1:48000_be_uja%2Fjq-n_0000000000) [n:127.0.0.1:48000_be_uja%2Fjq    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:48000/be_uja/jq",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 1
   [junit4]   2> 1549066 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1549067 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1549068 INFO  (zkCallback-967-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@783c947b name:ZooKeeperConnection Watcher:127.0.0.1:39006/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1549068 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1549069 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1549069 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1549071 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 1549071 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with stateFormat=2
   [junit4]   2> 1549071 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1549071 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1549072 INFO  (zkCallback-968-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1ab1815 name:ZooKeeperConnection Watcher:127.0.0.1:39006/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1549072 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1549073 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1549075 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x150c59130070006, likely client has closed socket
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 1549075 INFO  (OverseerStateUpdate-94792819541475332-127.0.0.1:48000_be_uja%2Fjq-n_0000000000) [n:127.0.0.1:48000_be_uja%2Fjq    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"create",
   [junit4]   2>   "name":"collection1",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "stateFormat":"2"} current state version: 1
   [junit4]   2> 1549075 INFO  (OverseerStateUpdate-94792819541475332-127.0.0.1:48000_be_uja%2Fjq-n_0000000000) [n:127.0.0.1:48000_be_uja%2Fjq    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 1549076 INFO  (zkCallback-965-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1549076 INFO  (zkCallback-967-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1549126 INFO  (coreZkRegister-3197-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq r:core_node1 x:collection1 c:control_collection s:shard1) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:48000/be_uja/jq/collection1/ and leader is http://127.0.0.1:48000/be_uja/jq/collection1/
   [junit4]   2> 1549126 INFO  (coreZkRegister-3197-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq r:core_node1 x:collection1 c:control_collection s:shard1) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:48000/be_uja/jq
   [junit4]   2> 1549126 INFO  (coreZkRegister-3197-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq r:core_node1 x:collection1 c:control_collection s:shard1) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 1549126 INFO  (coreZkRegister-3197-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq r:core_node1 x:collection1 c:control_collection s:shard1) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 1549126 INFO  (coreZkRegister-3197-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq r:core_node1 x:collection1 c:control_collection s:shard1) [n:127.0.0.1:48000_be_uja%2Fjq c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1549128 INFO  (OverseerStateUpdate-94792819541475332-127.0.0.1:48000_be_uja%2Fjq-n_0000000000) [n:127.0.0.1:48000_be_uja%2Fjq    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "roles":null,
   [junit4]   2>   "node_name":"127.0.0.1:48000_be_uja%2Fjq",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "base_url":"http://127.0.0.1:48000/be_uja/jq",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "state":"active"} current state version: 2
   [junit4]   2> 1549128 INFO  (OverseerStateUpdate-94792819541475332-127.0.0.1:48000_be_uja%2Fjq-n_0000000000) [n:127.0.0.1:48000_be_uja%2Fjq    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "roles":null,
   [junit4]   2>   "node_name":"127.0.0.1:48000_be_uja%2Fjq",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "base_url":"http://127.0.0.1:48000/be_uja/jq",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 1549129 INFO  (OverseerStateUpdate-94792819541475332-127.0.0.1:48000_be_uja%2Fjq-n_0000000000) [n:127.0.0.1:48000_be_uja%2Fjq    ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/collection1/state.json
   [junit4]   2> 1549233 INFO  (zkCallback-967-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1549233 INFO  (zkCallback-965-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1549344 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B8CE9E547BC18E6A-001/shard-1-001/cores/collection1
   [junit4]   2> 1549345 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B8CE9E547BC18E6A-001/shard-1-001
   [junit4]   2> 1549345 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 1549347 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@6ec88122{/be_uja/jq,null,AVAILABLE}
   [junit4]   2> 1549347 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.e.j.s.ServerConnector Started ServerConnector@5901bf61{HTTP/1.1}{127.0.0.1:50716}
   [junit4]   2> 1549347 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.e.j.s.Server Started @1552018ms
   [junit4]   2> 1549347 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B8CE9E547BC18E6A-001/shard-1-001/cores, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B8CE9E547BC18E6A-001/tempDir-001/jetty1, hostPort=50716, hostContext=/be_uja/jq, solrconfig=solrconfig.xml}
   [junit4]   2> 1549348 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@1fe91485
   [junit4]   2> 1549348 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B8CE9E547BC18E6A-001/shard-1-001/'
   [junit4]   2> 1549348 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1549348 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1549363 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1549363 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1549365 INFO  (zkCallback-969-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@351a5796 name:ZooKeeperConnection Watcher:127.0.0.1:39006/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1549365 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1549365 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1549366 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1549366 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B8CE9E547BC18E6A-001/shard-1-001/solr.xml
   [junit4]   2> 1549379 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B8CE9E547BC18E6A-001/shard-1-001/cores
   [junit4]   2> 1549379 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.CoreContainer New CoreContainer 42163332
   [junit4]   2> 1549379 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B8CE9E547BC18E6A-001/shard-1-001/]
   [junit4]   2> 1549380 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B8CE9E547BC18E6A-001/shard-1-001/lib
   [junit4]   2> 1549380 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B8CE9E547BC18E6A-001/shard-1-001/lib).
   [junit4]   2> 1549387 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1549388 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1549388 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1549388 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1549388 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:39006/solr
   [junit4]   2> 1549389 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 1549389 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1549389 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1549390 INFO  (zkCallback-971-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@4c9a7170 name:ZooKeeperConnection Watcher:127.0.0.1:39006 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1549391 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1549391 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1549393 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [n:127.0.0.1:50716_be_uja%2Fjq    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1549394 INFO  (zkCallback-972-thread-1-processing-n:127.0.0.1:50716_be_uja%2Fjq) [n:127.0.0.1:50716_be_uja%2Fjq    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5d48ee52 name:ZooKeeperConnection Watcher:127.0.0.1:39006/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1549394 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [n:127.0.0.1:50716_be_uja%2Fjq    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1549399 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [n:127.0.0.1:50716_be_uja%2Fjq    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1550405 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [n:127.0.0.1:50716_be_uja%2Fjq    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50716_be_uja%2Fjq
   [junit4]   2> 1550406 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [n:127.0.0.1:50716_be_uja%2Fjq    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:50716_be_uja%2Fjq
   [junit4]   2> 1550407 INFO  (zkCallback-972-thread-1-processing-n:127.0.0.1:50716_be_uja%2Fjq) [n:127.0.0.1:50716_be_uja%2Fjq    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1550407 INFO  (zkCallback-967-thread-1) [    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1550408 INFO  (zkCallback-965-thread-1-processing-n:127.0.0.1:48000_be_uja%2Fjq) [n:127.0.0.1:48000_be_uja%2Fjq    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1550410 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [n:127.0.0.1:50716_be_uja%2Fjq    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1550410 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [n:127.0.0.1:50716_be_uja%2Fjq    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94792819541475337-127.0.0.1:50716_be_uja%2Fjq-n_0000000001
   [junit4]   2> 1550411 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [n:127.0.0.1:50716_be_uja%2Fjq    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/94792819541475332-127.0.0.1:48000_be_uja%2Fjq-n_0000000000 to know if I could be the leader
   [junit4]   2> 1550422 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [n:127.0.0.1:50716_be_uja%2Fjq    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1550422 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [n:127.0.0.1:50716_be_uja%2Fjq    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1550423 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [n:127.0.0.1:50716_be_uja%2Fjq    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B8CE9E547BC18E6A-001/shard-1-001/cores
   [junit4]   2> 1550423 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [n:127.0.0.1:50716_be_uja%2Fjq    ] o.a.s.c.SolrCore Created CoreDescriptor: {config=solrconfig.xml, transient=false, loadOnStartup=true, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B8CE9E547BC18E6A-001/shard-1-001/cores/collection1, coreNodeName=, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B8CE9E547BC18E6A-001/shard-1-001/cores/collection1/, shard=, schema=schema.xml, configSetProperties=configsetprops.json, dataDir=data/, collection=collection1, name=collection1}
   [junit4]   2> 1550423 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [n:127.0.0.1:50716_be_uja%2Fjq    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_B8CE9E547BC18E6A-001/shard-1-001/cores/collection1/
   [junit4]   2> 1550423 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [n:127.0.0.1:50716_be_uja%2Fjq    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 1550425 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [n:127.0.0.1:50716_be_uja%2Fjq    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 1550425 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[B8CE9E547BC18E6A]) [n:127.0.0.1:50716_be_uja%2Fjq    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1550425 INFO  (coreLoadExecutor-3214-thread-1-processing-n:127.0.0.1:50716_be_uja%2Fjq) [n:127.0.0.1:50716_be_uja%2Fjq c:collection1   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 1550425 INFO  (coreLoadExecutor-3214-thread-1-processing-n:127.0.0.1:50716_be_uja%2Fjq) [n:127.0.0.1:50716_be_uja%2Fjq c:collection1   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1550427 INFO  (coreLoadExecutor-3214-thread-1-processing-n:127.0.0.1:50716_be_uja%2Fjq) [n:127.0.0.1:50716_be_uja%2Fjq c:collection1   x:collection1] o.a.s.c.ZkController Registering watch for collection collection1
   [junit4]   2> 1550427 INFO  (coreLoadExecutor-3214-thread-1-processing-n:127.0.0.1:50716_be_uja%2Fjq) [n:127.0.0.1:50716_be_uja%2Fjq c:collection1   x:collection1] o.a.s.c.c.ZkStateReader addZkWatch collection1
   [junit4]   2> 1550427 INFO  (OverseerStateUpdate-94792819541475332-127.0.0.1:48000_be_uja%2Fjq-n_0000000000) [n:127.0.0.1:48000_be_uja%2Fjq    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "roles":null,
   [junit4]   2>   "node_name":"127.0.0.1:50716_be_uja%2Fjq",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "base_url":"http://127.0.0.1:50716/be_uja/jq",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "state":"down"} current state version: 3
   [junit4]   2> 1550427 INFO  (OverseerStateUpdate-94792819541475332-127.0.0.1:48000_be_uja%2Fjq-n_0000000000) [n:127.0.0.1:48000_be_uja%2Fjq    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "roles":null,
   [junit4]   2>   "node_name":"127.0.0.1:50716_be_uja%2Fjq",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "base_url":"http://127.0.0.1:50716/be_uja/jq",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "state":"down"}
   [junit4]   2> 1550427 INFO  (coreLoadExecutor-3214-thread-1-processing-n:127.0.0.1:50716_be_uja%2Fjq) [n:127.0.0.1:50716_be_uja%2Fjq c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Add data for collection1 ver 0 
   [junit4]   2> 1550427 INFO  (OverseerStateUpdate-94792819541475332-127.0.0.1:48000_be_uja%2Fjq-n_0000000000) [n:127.0.0.1:48000_be_uja%2Fjq    ] o.a.s.c.o.ReplicaMutator Collection already exists with numShards=2
   [junit4]   2> 1550427 INFO  (coreLoadExecutor-3214-thread-1-processing-n:127.0.0.1:50716_be_uja%2Fjq) [n:127.0.0.1:50716_be_uja%2Fjq c:collection1   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 1550427 INFO  (OverseerStateUpdate-94792819541475332-127.0.0.1:48000_be_uja%2Fjq-n_0000000000) [n:127.0.0.1:48000_be_uja%2Fjq    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard2
   [junit4]   2> 1550530 INFO  (OverseerStateUpdate-94792819541475332-127.0.0.1:48000_be_uja%2Fjq-n_0000000000) [n:127.0.0.1:48000_be_uja%2Fjq    ] o.a.s.c.o.ZkStateWriter going to update_collection /collections/collection1/state.json version: 0
   [junit4]   2> 1550531 INFO  (zkCallback-972-thread-1-processing-n:127.0.0.1:50716_be_uja%2Fjq) [n:127.0.0.1:50716_be_uja%2Fjq    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json for collection collection1 has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1550531 INFO  (zkCallback-972-thread-1-processing-n:127.0.0.1:50716_be_uja%2Fjq) [n:127.0.0.1:50716_be_uja%2Fjq    ] o.a.s.c.c.ZkStateReader Updating data for collection1 from 0 to 1 
   [junit4]   2> 1551428 INFO  (coreLoadExecutor-3214-thread-1-processing-n:127.0.0.1:50716_be_uja%2Fjq) [n:127.0.0.1:50716_be_uja%2Fjq c:collection1   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1551428 INFO  (coreLoadExecutor-3214-thread-1-processing-n:127.0.0.1:50716_be_uja%2Fjq) [n:127.0.0.1:50716_be_uja%2Fjq c:collection1   x:collection1] o.a.s.c.ZkController Check for collection zkNode:collection1
   [junit4]   2> 1551428 INFO  (coreLoadExecutor-3214-thread-1-processing-n:127.0.0.1:50716_be_uja%2Fjq) [n:127.0.0.1:50716_be_uja%2Fjq c:collection1   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 1551428 INFO  (coreLoadEx

[...truncated too long message...]

/collections/collection1/leader_elect/shard2/election/94792899253764259-core_node6-n_0000000024
   [junit4]   2> 	2	/solr/collections/collection1/leader_elect/shard1/election/94792899253764256-core_node3-n_0000000026
   [junit4]   2> 	2	/solr/overseer_elect/election/94792899253764214-127.0.0.1:49811_ynoe%2Fp-n_0000000050
   [junit4]   2> 	2	/solr/overseer_elect/election/94792899253764172-127.0.0.1:58223_ynoe%2Fp-n_0000000029
   [junit4]   2> 	2	/solr/overseer_elect/election/94792899253764106-127.0.0.1:56562_ynoe%2Fp-n_0000000018
   [junit4]   2> 	2	/solr/overseer_elect/election/94792899253764136-127.0.0.1:60376_ynoe%2Fp-n_0000000011
   [junit4]   2> 	2	/solr/overseer_elect/election/94792899253764196-127.0.0.1:56077_ynoe%2Fp-n_0000000044
   [junit4]   2> 	2	/solr/collections/collection1/leader_elect/shard2/election/94792899253764133-core_node10-n_0000000004
   [junit4]   2> 	2	/solr/collections/collection1/leader_elect/shard1/election/94792899253764160-core_node3-n_0000000008
   [junit4]   2> 	2	/solr/collections/collection1/leader_elect/shard2/election/94792899253764115-core_node4-n_0000000001
   [junit4]   2> 	2	/solr/collections/collection1/leader_elect/shard1/election/94792899253764118-core_node5-n_0000000002
   [junit4]   2> 	2	/solr/overseer_elect/election/94792899253764127-127.0.0.1:41039_ynoe%2Fp-n_0000000008
   [junit4]   2> 	2	/solr/collections/collection1/leader_elect/shard2/election/94792899253764226-core_node8-n_0000000019
   [junit4]   2> 	2	/solr/collections/collection1/leader_elect/shard2/election/94792899253764139-core_node12-n_0000000005
   [junit4]   2> 	2	/solr/overseer_elect/election/94792899253764283-127.0.0.1:43518_ynoe%2Fp-n_0000000087
   [junit4]   2> 	2	/solr/overseer_elect/election/94792899253764124-127.0.0.1:56077_ynoe%2Fp-n_0000000007
   [junit4]   2> 	2	/solr/overseer_elect/election/94792899253764292-127.0.0.1:56077_ynoe%2Fp-n_0000000090
   [junit4]   2> 	2	/solr/overseer_elect/election/94792899253764220-127.0.0.1:58223_ynoe%2Fp-n_0000000052
   [junit4]   2> 	2	/solr/collections/collection1/leader_elect/shard1/election/94792899253764124-core_node7-n_0000000003
   [junit4]   2> 	2	/solr/collections/collection1/leader_elect/shard2/election/94792899253764178-core_node8-n_0000000012
   [junit4]   2> 
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2> 
   [junit4]   2> 	135	/solr/live_nodes
   [junit4]   2> 	69	/solr/collections
   [junit4]   2> 	29	/solr/overseer/collection-queue-work
   [junit4]   2> 	25	/solr/overseer/queue
   [junit4]   2> 	15	/solr/overseer/queue-work
   [junit4]   2> 
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=RollingRestartTest -Dtests.method=test -Dtests.seed=B8CE9E547BC18E6A -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=vi_VN -Dtests.timezone=GB -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] FAILURE  494s J2 | RollingRestartTest.test <<<
   [junit4]    > Throwable #1: java.lang.AssertionError: Unable to restart (#3): CloudJettyRunner [url=https://127.0.0.1:39313/ynoe/p/collection1]
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([B8CE9E547BC18E6A:309AA18ED53DE392]:0)
   [junit4]    > 	at org.apache.solr.cloud.RollingRestartTest.restartWithRolesTest(RollingRestartTest.java:104)
   [junit4]    > 	at org.apache.solr.cloud.RollingRestartTest.test(RollingRestartTest.java:53)
   [junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:963)
   [junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:938)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 3256946 INFO  (SUITE-RollingRestartTest-seed#[B8CE9E547BC18E6A]-worker) [n:127.0.0.1:38744_ynoe%2Fp c:collection1 s:shard2 r:core_node16 x:collection1] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.RollingRestartTest_B8CE9E547BC18E6A-001
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene54): {}, docValues:{}, sim=DefaultSimilarity, locale=vi_VN, timezone=GB
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_72 (64-bit)/cpus=4,threads=1,free=227910440,total=528482304
   [junit4]   2> NOTE: All tests run in this JVM: [TestIndexingPerformance, TermVectorComponentDistributedTest, TestCloudManagedSchemaConcurrent, DirectSolrConnectionTest, TestReRankQParserPlugin, RegexBoostProcessorTest, TestElisionMultitermQuery, TestDefaultSimilarityFactory, SuggesterTSTTest, TestDefaultSearchFieldResource, XmlUpdateRequestHandlerTest, DocValuesMissingTest, TestRebalanceLeaders, CoreMergeIndexesAdminHandlerTest, TestCollationField, TestOrdValues, TestEmbeddedSolrServerConstructors, TestReload, TestFieldTypeCollectionResource, SimpleFacetsTest, LeaderInitiatedRecoveryOnShardRestartTest, SuggesterTest, StatsComponentTest, HLLSerializationTest, DocValuesMultiTest, TestCloudInspectUtil, TestCoreDiscovery, CoreAdminHandlerTest, HdfsWriteToMultipleCollectionsTest, SparseHLLTest, DistribDocExpirationUpdateProcessorTest, DirectUpdateHandlerOptimizeTest, DeleteLastCustomShardedReplicaTest, RulesTest, RuleEngineTest, TestStressRecovery, TestComplexPhraseQParserPlugin, UnloadDistributedZkTest, HttpPartitionTest, TestFuzzyAnalyzedSuggestions, InfoHandlerTest, TestSort, UpdateParamsTest, AliasIntegrationTest, FieldMutatingUpdateProcessorTest, TestSmileRequest, PrimitiveFieldTypeTest, TestPHPSerializedResponseWriter, DistributedQueryComponentCustomSortTest, TestExceedMaxTermLength, TestIndexSearcher, TestSystemIdResolver, TestFastLRUCache, SharedFSAutoReplicaFailoverUtilsTest, SearchHandlerTest, TestScoreJoinQPNoScore, TestJmxIntegration, TestSolrIndexConfig, TestChildDocTransformer, CollectionTooManyReplicasTest, LukeRequestHandlerTest, RecoveryZkTest, CachingDirectoryFactoryTest, SampleTest, TestSweetSpotSimilarityFactory, TestNRTOpen, TestSolrDeletionPolicy1, FullSolrCloudDistribCmdsTest, ChangedSchemaMergeTest, TestDynamicLoading, TestHashQParserPlugin, VMParamsZkACLAndCredentialsProvidersTest, TestConfigSets, TestSchemaVersionResource, TestRequestStatusCollectionAPI, ExplicitHLLTest, TestBulkSchemaAPI, TestSchemaNameResource, SuggesterWFSTTest, TestIntervalFaceting, DistributedFacetPivotLargeTest, TestLeaderElectionZkExpiry, DistributedExpandComponentTest, TestSimpleQParserPlugin, BlockDirectoryTest, TestRecovery, TestXIncludeConfig, QueryParsingTest, BasicDistributedZk2Test, OpenCloseCoreStressTest, AutoCommitTest, TestStressReorder, HardAutoCommitTest, DistributedTermsComponentTest, TestRangeQuery, SolrCoreTest, TestGroupingSearch, TestFunctionQuery, DirectUpdateHandlerTest, TestBadConfig, ShowFileRequestHandlerTest, CurrencyFieldOpenExchangeTest, CurrencyFieldXmlFileTest, SimplePostToolTest, SignatureUpdateProcessorFactoryTest, SpellCheckCollatorTest, TestStressLucene, SpatialFilterTest, SchemaVersionSpecificBehaviorTest, StatelessScriptUpdateProcessorFactoryTest, SortByFunctionTest, SolrInfoMBeanTest, CacheHeaderTest, TestQueryUtils, TestReversedWildcardFilterFactory, TestQueryTypes, TermsComponentTest, PathHierarchyTokenizerFactoryTest, RequiredFieldsTest, TestSolrQueryParser, IndexSchemaRuntimeFieldTest, UpdateRequestProcessorFactoryTest, TestCSVResponseWriter, BinaryUpdateRequestHandlerTest, CSVRequestHandlerTest, BadComponentTest, MultiTermTest, TestSearchPerf, OutputWriterTest, ExternalFileFieldSortTest, TestPostingsSolrHighlighter, TestLuceneMatchVersion, TestCharFilters, TestCodecSupport, TestDFRSimilarityFactory, TestIBSimilarityFactory, TestFastOutputStream, ScriptEngineTest, URLClassifyProcessorTest, DateMathParserTest, DOMUtilTest, RAMDirectoryFactoryTest, ClusterStateTest, TestLRUCache, TestUtils, ZkNodePropsTest, SliceStateTest, SystemInfoHandlerTest, UUIDFieldTest, FileUtilsTest, CircularListTest, CursorPagingTest, DistributedIntervalFacetingTest, SolrTestCaseJ4Test, TestCursorMarkWithoutUniqueKey, TestHighlightDedupGrouping, TestSimpleTrackingShardHandler, ActionThrottleTest, AsyncMigrateRouteKeyTest, CollectionStateFormat2Test, ConnectionManagerTest, DistribCursorPagingTest, DistributedQueueTest, MultiThreadedOCPTest, ReplicationFactorTest, RollingRestartTest]
   [junit4] Completed [442/543] on J2 in 493.75s, 1 test, 1 failure <<< FAILURES!

[...truncated 331 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 543 suites (7 ignored), 2147 tests, 1 error, 1 failure, 31 ignored (18 assumptions) [seed: B8CE9E547BC18E6A]

Total time: 286 minutes 7 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

RE: [JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1003 - Still Failing

[Martin Gainty <mg...@hotmail.com>]

a few discoveries
almost all of the Abstract Classes and Interfaces for org.apache.lucene.analysis.hunspell.Dictionary have no impl classes for either abstract classes and the interfaces
this causes havoc in the lucene-analyzers-common folder for org.apache.lucene.analysis.hunspell.StemmerTestBase testcaseNotably these Impl Classes were missing in Dictionary:
public class PathImpl implements java.nio.file.Path     
public class ComparatorImpl<T> implements java.util.Comparator<T>
I dont know if these were you the specific issue you came upon but I found ivy's testcase engine lacking
currently re-tasking the more mission critical analyzers to build with maven pom.xml for these reasons:
-easier to unit test-use dependencyManagement, pluginManagement to inherit dependencies and plugins from grandfather pom.xml
-flow follows maven lifecycle instead of bouncing all over the place as ivy bounces from grandchild to grandfather
to sibling...absolutely impossible to follow ivy workflow structure with no lifecycle for the build execution plan to followMavens lifecycle is executed always in the order
Grandfather--> calls to Children --> calls to Grandchildren
each of the grandchildren are executed in specified <module>child1</module> <module>child2</module> order

Are these the failures/errors you are seeing David?
Martin


From: david.w.smiley@gmail.com
Date: Sun, 1 Nov 2015 03:53:24 +0000
Subject: Re: [JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1003 - Still Failing
To: dev@lucene.apache.org

I created an issue: https://issues.apache.org/jira/browse/LUCENE-6873
On Sat, Oct 31, 2015 at 7:52 PM Apache Jenkins Server <je...@builds.apache.org> wrote:
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1003/



1 tests failed.

FAILED:  org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest.testRandom {#11 seed=[D30126801503C858:4E8BBD0E05B60CEF]}



Error Message:

maxY must be >= minY: -23.184599227870876 to -23.18459922787088



Stack Trace:

com.spatial4j.core.exception.InvalidShapeException: maxY must be >= minY: -23.184599227870876 to -23.18459922787088

        at __randomizedtesting.SeedInfo.seed([D30126801503C858:4E8BBD0E05B60CEF]:0)

        at com.spatial4j.core.context.SpatialContext.makeRectangle(SpatialContext.java:218)

        at org.apache.lucene.spatial.SpatialTestCase.randomRectangle(SpatialTestCase.java:175)

        at org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest.queryHeatmapRecursive(HeatmapFacetCounterTest.java:207)

        at org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest.queryHeatmapRecursive(HeatmapFacetCounterTest.java:207)

        at org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest.queryHeatmapRecursive(HeatmapFacetCounterTest.java:207)

        at org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest.testRandom(HeatmapFacetCounterTest.java:169)

        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

        at java.lang.reflect.Method.invoke(Method.java:606)

        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1660)

        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:866)

        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:902)

        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:916)

        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)

        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)

        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)

        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)

        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)

        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)

        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)

        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)

        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)

        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:875)

        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:777)

        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:811)

        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:822)

        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)

        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)

        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)

        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)

        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)

        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)

        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)

        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)

        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)

        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)

        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)

        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)

        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)

        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)

        at java.lang.Thread.run(Thread.java:745)









Build Log:

[...truncated 8808 lines...]

   [junit4] Suite: org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest

   [junit4]   2> ??? 01, 2015 5:49:58 ?? org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after

   [junit4]   2> INFO: Validated 15 cells, 5 non-zero

   [junit4]   2> ??? 01, 2015 5:49:58 ?? org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after

   [junit4]   2> INFO: Validated 13 cells, 8 non-zero

   [junit4]   2> ??? 01, 2015 5:49:58 ?? org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after

   [junit4]   2> INFO: Validated 21 cells, 7 non-zero

   [junit4]   2> ??? 01, 2015 5:49:58 ?? org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after

   [junit4]   2> INFO: Validated 2 cells, 0 non-zero

   [junit4]   2> ??? 01, 2015 5:49:58 ?? org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after

   [junit4]   2> INFO: Validated 21 cells, 10 non-zero

   [junit4]   2> ??? 01, 2015 5:49:58 ?? org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after

   [junit4]   2> INFO: Validated 5 cells, 5 non-zero

   [junit4]   2> ??? 01, 2015 5:49:58 ?? org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after

   [junit4]   2> INFO: Validated 10 cells, 6 non-zero

   [junit4]   2> ??? 01, 2015 5:49:59 ?? org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after

   [junit4]   2> INFO: Validated 8 cells, 7 non-zero

   [junit4]   2> ??? 01, 2015 5:49:59 ?? org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after

   [junit4]   2> INFO: Validated 8 cells, 7 non-zero

   [junit4]   2> ??? 01, 2015 5:49:59 ?? org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after

   [junit4]   2> INFO: Validated 8 cells, 7 non-zero

   [junit4]   2> ??? 01, 2015 5:49:59 ?? org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after

   [junit4]   2> INFO: Validated 12 cells, 8 non-zero

   [junit4]   2> ??? 01, 2015 5:49:59 ?? org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after

   [junit4]   2> INFO: Validated 12 cells, 8 non-zero

   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.

   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=HeatmapFacetCounterTest -Dtests.method=testRandom -Dtests.seed=D30126801503C858 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=el -Dtests.timezone=Asia/Qyzylorda -Dtests.asserts=true -Dtests.file.encoding=US-ASCII

   [junit4] ERROR   0.04s J1 | HeatmapFacetCounterTest.testRandom {#11 seed=[D30126801503C858:4E8BBD0E05B60CEF]} <<<

   [junit4]    > Throwable #1: com.spatial4j.core.exception.InvalidShapeException: maxY must be >= minY: -23.184599227870876 to -23.18459922787088

   [junit4]    >        at __randomizedtesting.SeedInfo.seed([D30126801503C858:4E8BBD0E05B60CEF]:0)

   [junit4]    >        at com.spatial4j.core.context.SpatialContext.makeRectangle(SpatialContext.java:218)

   [junit4]    >        at org.apache.lucene.spatial.SpatialTestCase.randomRectangle(SpatialTestCase.java:175)

   [junit4]    >        at org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest.queryHeatmapRecursive(HeatmapFacetCounterTest.java:207)

   [junit4]    >        at org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest.queryHeatmapRecursive(HeatmapFacetCounterTest.java:207)

   [junit4]    >        at org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest.queryHeatmapRecursive(HeatmapFacetCounterTest.java:207)

   [junit4]    >        at org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest.testRandom(HeatmapFacetCounterTest.java:169)

   [junit4]    >        at java.lang.Thread.run(Thread.java:745)

   [junit4]   2> ??? 01, 2015 5:49:59 ?? org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after

   [junit4]   2> INFO: Validated 8 cells, 7 non-zero

   [junit4]   2> ??? 01, 2015 5:49:59 ?? org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after

   [junit4]   2> INFO: Validated 27 cells, 8 non-zero

   [junit4]   2> ??? 01, 2015 5:49:59 ?? org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after

   [junit4]   2> INFO: Validated 8 cells, 8 non-zero

   [junit4]   2> ??? 01, 2015 5:49:59 ?? org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after

   [junit4]   2> INFO: Validated 0 cells, 0 non-zero

   [junit4]   2> ??? 01, 2015 5:50:00 ?? org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after

   [junit4]   2> INFO: Validated 12 cells, 8 non-zero

   [junit4]   2> ??? 01, 2015 5:50:00 ?? org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after

   [junit4]   2> INFO: Validated 10 cells, 9 non-zero

   [junit4]   2> ??? 01, 2015 5:50:00 ?? org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after

   [junit4]   2> INFO: Validated 12 cells, 6 non-zero

   [junit4]   2> ??? 01, 2015 5:50:00 ?? org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after

   [junit4]   2> INFO: Validated 5 cells, 4 non-zero

   [junit4]   2> ??? 01, 2015 5:50:00 ?? org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after

   [junit4]   2> INFO: Validated 11 cells, 9 non-zero

   [junit4]   2> ??? 01, 2015 5:50:00 ?? org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after

   [junit4]   2> INFO: Validated 0 cells, 0 non-zero

   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/spatial/test/J1/temp/lucene.spatial.prefix.HeatmapFacetCounterTest_D30126801503C858-001

   [junit4]   2> NOTE: test params are: codec=Lucene54, sim=RandomSimilarityProvider(queryNorm=true,coord=no): {}, locale=el, timezone=Asia/Qyzylorda

   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_72 (64-bit)/cpus=4,threads=1,free=240746280,total=342884352

   [junit4]   2> NOTE: All tests run in this JVM: [TestPointVectorStrategy, TestTestFramework, JtsPolygonTest, HeatmapFacetCounterTest]

   [junit4] Completed [10/23] on J1 in 2.24s, 22 tests, 1 error <<< FAILURES!



[...truncated 50 lines...]

BUILD FAILED

/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:

/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:

/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:

/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build.xml:471: The following error occurred while executing this line:

/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:2248: The following error occurred while executing this line:

/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/module-build.xml:58: The following error occurred while executing this line:

/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:

/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 23 suites, 235 tests, 1 error, 11 ignored (10 assumptions) [seed: D30126801503C858]



Total time: 201 minutes 15 seconds

Build step 'Invoke Ant' marked build as failure

Archiving artifacts

Recording test results

Email was triggered for: Failure - Any

Sending email for trigger: Failure - Any







---------------------------------------------------------------------

To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org

For additional commands, e-mail: dev-help@lucene.apache.org-- 
Lucene/Solr Search Committer, Consultant, Developer, Author, SpeakerLinkedIn: http://linkedin.com/in/davidwsmiley | Book: http://www.solrenterprisesearchserver.com 		 	   		  

Re: [JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1003 - Still Failing

["david.w.smiley@gmail.com" <da...@gmail.com>]

I created an issue: https://issues.apache.org/jira/browse/LUCENE-6873

On Sat, Oct 31, 2015 at 7:52 PM Apache Jenkins Server <
jenkins@builds.apache.org> wrote:

> Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1003/
>
> 1 tests failed.
> FAILED:
> org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest.testRandom {#11
> seed=[D30126801503C858:4E8BBD0E05B60CEF]}
>
> Error Message:
> maxY must be >= minY: -23.184599227870876 to -23.18459922787088
>
> Stack Trace:
> com.spatial4j.core.exception.InvalidShapeException: maxY must be >= minY:
> -23.184599227870876 to -23.18459922787088
>         at
> __randomizedtesting.SeedInfo.seed([D30126801503C858:4E8BBD0E05B60CEF]:0)
>         at
> com.spatial4j.core.context.SpatialContext.makeRectangle(SpatialContext.java:218)
>         at
> org.apache.lucene.spatial.SpatialTestCase.randomRectangle(SpatialTestCase.java:175)
>         at
> org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest.queryHeatmapRecursive(HeatmapFacetCounterTest.java:207)
>         at
> org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest.queryHeatmapRecursive(HeatmapFacetCounterTest.java:207)
>         at
> org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest.queryHeatmapRecursive(HeatmapFacetCounterTest.java:207)
>         at
> org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest.testRandom(HeatmapFacetCounterTest.java:169)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>         at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>         at java.lang.reflect.Method.invoke(Method.java:606)
>         at
> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1660)
>         at
> com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:866)
>         at
> com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:902)
>         at
> com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:916)
>         at
> org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
>         at
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
>         at
> org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
>         at
> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
>         at
> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
>         at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>         at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
>         at
> com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
>         at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
>         at
> com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:875)
>         at
> com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:777)
>         at
> com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:811)
>         at
> com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:822)
>         at
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
>         at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>         at
> org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
>         at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
>         at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
>         at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>         at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>         at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>         at
> org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
>         at
> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
>         at
> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
>         at
> org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
>         at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>         at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
>         at java.lang.Thread.run(Thread.java:745)
>
>
>
>
> Build Log:
> [...truncated 8808 lines...]
>    [junit4] Suite: org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest
>    [junit4]   2> ??? 01, 2015 5:49:58 ??
> org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after
>    [junit4]   2> INFO: Validated 15 cells, 5 non-zero
>    [junit4]   2> ??? 01, 2015 5:49:58 ??
> org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after
>    [junit4]   2> INFO: Validated 13 cells, 8 non-zero
>    [junit4]   2> ??? 01, 2015 5:49:58 ??
> org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after
>    [junit4]   2> INFO: Validated 21 cells, 7 non-zero
>    [junit4]   2> ??? 01, 2015 5:49:58 ??
> org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after
>    [junit4]   2> INFO: Validated 2 cells, 0 non-zero
>    [junit4]   2> ??? 01, 2015 5:49:58 ??
> org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after
>    [junit4]   2> INFO: Validated 21 cells, 10 non-zero
>    [junit4]   2> ??? 01, 2015 5:49:58 ??
> org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after
>    [junit4]   2> INFO: Validated 5 cells, 5 non-zero
>    [junit4]   2> ??? 01, 2015 5:49:58 ??
> org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after
>    [junit4]   2> INFO: Validated 10 cells, 6 non-zero
>    [junit4]   2> ??? 01, 2015 5:49:59 ??
> org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after
>    [junit4]   2> INFO: Validated 8 cells, 7 non-zero
>    [junit4]   2> ??? 01, 2015 5:49:59 ??
> org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after
>    [junit4]   2> INFO: Validated 8 cells, 7 non-zero
>    [junit4]   2> ??? 01, 2015 5:49:59 ??
> org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after
>    [junit4]   2> INFO: Validated 8 cells, 7 non-zero
>    [junit4]   2> ??? 01, 2015 5:49:59 ??
> org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after
>    [junit4]   2> INFO: Validated 12 cells, 8 non-zero
>    [junit4]   2> ??? 01, 2015 5:49:59 ??
> org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after
>    [junit4]   2> INFO: Validated 12 cells, 8 non-zero
>    [junit4]   2> NOTE: download the large Jenkins line-docs file by
> running 'ant get-jenkins-line-docs' in the lucene directory.
>    [junit4]   2> NOTE: reproduce with: ant test
> -Dtestcase=HeatmapFacetCounterTest -Dtests.method=testRandom
> -Dtests.seed=D30126801503C858 -Dtests.multiplier=2 -Dtests.nightly=true
> -Dtests.slow=true
> -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt
> -Dtests.locale=el -Dtests.timezone=Asia/Qyzylorda -Dtests.asserts=true
> -Dtests.file.encoding=US-ASCII
>    [junit4] ERROR   0.04s J1 | HeatmapFacetCounterTest.testRandom {#11
> seed=[D30126801503C858:4E8BBD0E05B60CEF]} <<<
>    [junit4]    > Throwable #1:
> com.spatial4j.core.exception.InvalidShapeException: maxY must be >= minY:
> -23.184599227870876 to -23.18459922787088
>    [junit4]    >        at
> __randomizedtesting.SeedInfo.seed([D30126801503C858:4E8BBD0E05B60CEF]:0)
>    [junit4]    >        at
> com.spatial4j.core.context.SpatialContext.makeRectangle(SpatialContext.java:218)
>    [junit4]    >        at
> org.apache.lucene.spatial.SpatialTestCase.randomRectangle(SpatialTestCase.java:175)
>    [junit4]    >        at
> org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest.queryHeatmapRecursive(HeatmapFacetCounterTest.java:207)
>    [junit4]    >        at
> org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest.queryHeatmapRecursive(HeatmapFacetCounterTest.java:207)
>    [junit4]    >        at
> org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest.queryHeatmapRecursive(HeatmapFacetCounterTest.java:207)
>    [junit4]    >        at
> org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest.testRandom(HeatmapFacetCounterTest.java:169)
>    [junit4]    >        at java.lang.Thread.run(Thread.java:745)
>    [junit4]   2> ??? 01, 2015 5:49:59 ??
> org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after
>    [junit4]   2> INFO: Validated 8 cells, 7 non-zero
>    [junit4]   2> ??? 01, 2015 5:49:59 ??
> org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after
>    [junit4]   2> INFO: Validated 27 cells, 8 non-zero
>    [junit4]   2> ??? 01, 2015 5:49:59 ??
> org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after
>    [junit4]   2> INFO: Validated 8 cells, 8 non-zero
>    [junit4]   2> ??? 01, 2015 5:49:59 ??
> org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after
>    [junit4]   2> INFO: Validated 0 cells, 0 non-zero
>    [junit4]   2> ??? 01, 2015 5:50:00 ??
> org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after
>    [junit4]   2> INFO: Validated 12 cells, 8 non-zero
>    [junit4]   2> ??? 01, 2015 5:50:00 ??
> org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after
>    [junit4]   2> INFO: Validated 10 cells, 9 non-zero
>    [junit4]   2> ??? 01, 2015 5:50:00 ??
> org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after
>    [junit4]   2> INFO: Validated 12 cells, 6 non-zero
>    [junit4]   2> ??? 01, 2015 5:50:00 ??
> org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after
>    [junit4]   2> INFO: Validated 5 cells, 4 non-zero
>    [junit4]   2> ??? 01, 2015 5:50:00 ??
> org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after
>    [junit4]   2> INFO: Validated 11 cells, 9 non-zero
>    [junit4]   2> ??? 01, 2015 5:50:00 ??
> org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after
>    [junit4]   2> INFO: Validated 0 cells, 0 non-zero
>    [junit4]   2> NOTE: leaving temporary files on disk at:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/spatial/test/J1/temp/lucene.spatial.prefix.HeatmapFacetCounterTest_D30126801503C858-001
>    [junit4]   2> NOTE: test params are: codec=Lucene54,
> sim=RandomSimilarityProvider(queryNorm=true,coord=no): {}, locale=el,
> timezone=Asia/Qyzylorda
>    [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation
> 1.7.0_72 (64-bit)/cpus=4,threads=1,free=240746280,total=342884352
>    [junit4]   2> NOTE: All tests run in this JVM:
> [TestPointVectorStrategy, TestTestFramework, JtsPolygonTest,
> HeatmapFacetCounterTest]
>    [junit4] Completed [10/23] on J1 in 2.24s, 22 tests, 1 error <<<
> FAILURES!
>
> [...truncated 50 lines...]
> BUILD FAILED
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792:
> The following error occurred while executing this line:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729:
> The following error occurred while executing this line:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59:
> The following error occurred while executing this line:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build.xml:471:
> The following error occurred while executing this line:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:2248:
> The following error occurred while executing this line:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/module-build.xml:58:
> The following error occurred while executing this line:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452:
> The following error occurred while executing this line:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006:
> There were test failures: 23 suites, 235 tests, 1 error, 11 ignored (10
> assumptions) [seed: D30126801503C858]
>
> Total time: 201 minutes 15 seconds
> Build step 'Invoke Ant' marked build as failure
> Archiving artifacts
> Recording test results
> Email was triggered for: Failure - Any
> Sending email for trigger: Failure - Any
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org

-- 
Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
LinkedIn: http://linkedin.com/in/davidwsmiley | Book:
http://www.solrenterprisesearchserver.com

[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1003 - Still Failing

[Apache Jenkins Server <je...@builds.apache.org>]

Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1003/

1 tests failed.
FAILED:  org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest.testRandom {#11 seed=[D30126801503C858:4E8BBD0E05B60CEF]}

Error Message:
maxY must be >= minY: -23.184599227870876 to -23.18459922787088

Stack Trace:
com.spatial4j.core.exception.InvalidShapeException: maxY must be >= minY: -23.184599227870876 to -23.18459922787088
	at __randomizedtesting.SeedInfo.seed([D30126801503C858:4E8BBD0E05B60CEF]:0)
	at com.spatial4j.core.context.SpatialContext.makeRectangle(SpatialContext.java:218)
	at org.apache.lucene.spatial.SpatialTestCase.randomRectangle(SpatialTestCase.java:175)
	at org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest.queryHeatmapRecursive(HeatmapFacetCounterTest.java:207)
	at org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest.queryHeatmapRecursive(HeatmapFacetCounterTest.java:207)
	at org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest.queryHeatmapRecursive(HeatmapFacetCounterTest.java:207)
	at org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest.testRandom(HeatmapFacetCounterTest.java:169)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1660)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:866)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:902)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:916)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:875)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:777)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:811)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:822)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 8808 lines...]
   [junit4] Suite: org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest
   [junit4]   2> ??? 01, 2015 5:49:58 ?? org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after
   [junit4]   2> INFO: Validated 15 cells, 5 non-zero
   [junit4]   2> ??? 01, 2015 5:49:58 ?? org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after
   [junit4]   2> INFO: Validated 13 cells, 8 non-zero
   [junit4]   2> ??? 01, 2015 5:49:58 ?? org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after
   [junit4]   2> INFO: Validated 21 cells, 7 non-zero
   [junit4]   2> ??? 01, 2015 5:49:58 ?? org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after
   [junit4]   2> INFO: Validated 2 cells, 0 non-zero
   [junit4]   2> ??? 01, 2015 5:49:58 ?? org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after
   [junit4]   2> INFO: Validated 21 cells, 10 non-zero
   [junit4]   2> ??? 01, 2015 5:49:58 ?? org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after
   [junit4]   2> INFO: Validated 5 cells, 5 non-zero
   [junit4]   2> ??? 01, 2015 5:49:58 ?? org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after
   [junit4]   2> INFO: Validated 10 cells, 6 non-zero
   [junit4]   2> ??? 01, 2015 5:49:59 ?? org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after
   [junit4]   2> INFO: Validated 8 cells, 7 non-zero
   [junit4]   2> ??? 01, 2015 5:49:59 ?? org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after
   [junit4]   2> INFO: Validated 8 cells, 7 non-zero
   [junit4]   2> ??? 01, 2015 5:49:59 ?? org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after
   [junit4]   2> INFO: Validated 8 cells, 7 non-zero
   [junit4]   2> ??? 01, 2015 5:49:59 ?? org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after
   [junit4]   2> INFO: Validated 12 cells, 8 non-zero
   [junit4]   2> ??? 01, 2015 5:49:59 ?? org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after
   [junit4]   2> INFO: Validated 12 cells, 8 non-zero
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=HeatmapFacetCounterTest -Dtests.method=testRandom -Dtests.seed=D30126801503C858 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=el -Dtests.timezone=Asia/Qyzylorda -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
   [junit4] ERROR   0.04s J1 | HeatmapFacetCounterTest.testRandom {#11 seed=[D30126801503C858:4E8BBD0E05B60CEF]} <<<
   [junit4]    > Throwable #1: com.spatial4j.core.exception.InvalidShapeException: maxY must be >= minY: -23.184599227870876 to -23.18459922787088
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([D30126801503C858:4E8BBD0E05B60CEF]:0)
   [junit4]    > 	at com.spatial4j.core.context.SpatialContext.makeRectangle(SpatialContext.java:218)
   [junit4]    > 	at org.apache.lucene.spatial.SpatialTestCase.randomRectangle(SpatialTestCase.java:175)
   [junit4]    > 	at org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest.queryHeatmapRecursive(HeatmapFacetCounterTest.java:207)
   [junit4]    > 	at org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest.queryHeatmapRecursive(HeatmapFacetCounterTest.java:207)
   [junit4]    > 	at org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest.queryHeatmapRecursive(HeatmapFacetCounterTest.java:207)
   [junit4]    > 	at org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest.testRandom(HeatmapFacetCounterTest.java:169)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> ??? 01, 2015 5:49:59 ?? org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after
   [junit4]   2> INFO: Validated 8 cells, 7 non-zero
   [junit4]   2> ??? 01, 2015 5:49:59 ?? org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after
   [junit4]   2> INFO: Validated 27 cells, 8 non-zero
   [junit4]   2> ??? 01, 2015 5:49:59 ?? org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after
   [junit4]   2> INFO: Validated 8 cells, 8 non-zero
   [junit4]   2> ??? 01, 2015 5:49:59 ?? org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after
   [junit4]   2> INFO: Validated 0 cells, 0 non-zero
   [junit4]   2> ??? 01, 2015 5:50:00 ?? org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after
   [junit4]   2> INFO: Validated 12 cells, 8 non-zero
   [junit4]   2> ??? 01, 2015 5:50:00 ?? org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after
   [junit4]   2> INFO: Validated 10 cells, 9 non-zero
   [junit4]   2> ??? 01, 2015 5:50:00 ?? org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after
   [junit4]   2> INFO: Validated 12 cells, 6 non-zero
   [junit4]   2> ??? 01, 2015 5:50:00 ?? org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after
   [junit4]   2> INFO: Validated 5 cells, 4 non-zero
   [junit4]   2> ??? 01, 2015 5:50:00 ?? org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after
   [junit4]   2> INFO: Validated 11 cells, 9 non-zero
   [junit4]   2> ??? 01, 2015 5:50:00 ?? org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest after
   [junit4]   2> INFO: Validated 0 cells, 0 non-zero
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/spatial/test/J1/temp/lucene.spatial.prefix.HeatmapFacetCounterTest_D30126801503C858-001
   [junit4]   2> NOTE: test params are: codec=Lucene54, sim=RandomSimilarityProvider(queryNorm=true,coord=no): {}, locale=el, timezone=Asia/Qyzylorda
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_72 (64-bit)/cpus=4,threads=1,free=240746280,total=342884352
   [junit4]   2> NOTE: All tests run in this JVM: [TestPointVectorStrategy, TestTestFramework, JtsPolygonTest, HeatmapFacetCounterTest]
   [junit4] Completed [10/23] on J1 in 2.24s, 22 tests, 1 error <<< FAILURES!

[...truncated 50 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build.xml:471: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:2248: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/module-build.xml:58: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 23 suites, 235 tests, 1 error, 11 ignored (10 assumptions) [seed: D30126801503C858]

Total time: 201 minutes 15 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1002 - Still Failing

[Apache Jenkins Server <je...@builds.apache.org>]

Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1002/

5 tests failed.
FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Error from server at https://127.0.0.1:36938/xclx/ta: Could not load collection from ZK:acollectionafterbaddelete

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:36938/xclx/ta: Could not load collection from ZK:acollectionafterbaddelete
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.makeRequest(CollectionsAPIDistributedZkTest.java:302)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testErrorHandling(CollectionsAPIDistributedZkTest.java:419)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test(CollectionsAPIDistributedZkTest.java:169)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1660)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:866)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:902)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:916)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:963)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:938)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:875)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:777)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:811)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:822)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.CollectionsAPIDistributedZkTest

Error Message:
1 thread leaked from SUITE scope at org.apache.solr.cloud.CollectionsAPIDistributedZkTest:     1) Thread[id=18893, name=searcherExecutor-6813-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.cloud.CollectionsAPIDistributedZkTest: 
   1) Thread[id=18893, name=searcherExecutor-6813-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
	at __randomizedtesting.SeedInfo.seed([23AE55F0085E3EA5]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.CollectionsAPIDistributedZkTest

Error Message:
There are still zombie threads that couldn't be terminated:    1) Thread[id=18893, name=searcherExecutor-6813-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   1) Thread[id=18893, name=searcherExecutor-6813-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
	at __randomizedtesting.SeedInfo.seed([23AE55F0085E3EA5]:0)


FAILED:  org.apache.solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR

Error Message:
Captured an uncaught exception in thread: Thread[id=8681, name=coreZkRegister-1921-thread-1, state=RUNNABLE, group=TGRP-LeaderInitiatedRecoveryOnShardRestartTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=8681, name=coreZkRegister-1921-thread-1, state=RUNNABLE, group=TGRP-LeaderInitiatedRecoveryOnShardRestartTest]
Caused by: java.lang.AssertionError
	at __randomizedtesting.SeedInfo.seed([23AE55F0085E3EA5]:0)
	at org.apache.solr.cloud.ZkController.updateLeaderInitiatedRecoveryState(ZkController.java:2133)
	at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:434)
	at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)
	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)
	at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:346)
	at org.apache.solr.cloud.ZkController.joinElection(ZkController.java:1113)
	at org.apache.solr.cloud.ZkController.register(ZkController.java:926)
	at org.apache.solr.cloud.ZkController.register(ZkController.java:881)
	at org.apache.solr.core.ZkContainer$2.run(ZkContainer.java:183)
	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.handler.TestReplicationHandlerBackup.doTestBackup

Error Message:
Failed to create backup

Stack Trace:
java.lang.AssertionError: Failed to create backup
	at __randomizedtesting.SeedInfo.seed([23AE55F0085E3EA5:622575952FE0CDEA]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.solr.handler.CheckBackupStatus.fetchStatus(CheckBackupStatus.java:51)
	at org.apache.solr.handler.TestReplicationHandlerBackup.doTestBackup(TestReplicationHandlerBackup.java:197)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1660)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:866)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:902)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:916)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:875)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:777)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:811)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:822)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 10600 lines...]
   [junit4] Suite: org.apache.solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_23AE55F0085E3EA5-001/init-core-data-001
   [junit4]   2> 739605 INFO  (SUITE-LeaderInitiatedRecoveryOnShardRestartTest-seed#[23AE55F0085E3EA5]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /_uv/b
   [junit4]   2> 739610 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 739610 INFO  (Thread-5635) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 739610 INFO  (Thread-5635) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 739710 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.ZkTestServer start zk server on port:48193
   [junit4]   2> 739711 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 739712 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 739716 INFO  (zkCallback-691-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@791b2bc1 name:ZooKeeperConnection Watcher:127.0.0.1:48193 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 739716 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 739716 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 739716 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 739719 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 739720 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 739721 INFO  (zkCallback-692-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@3d493df name:ZooKeeperConnection Watcher:127.0.0.1:48193/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 739721 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 739721 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 739721 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 739724 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 739726 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 739727 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 739729 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 739729 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 739731 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 739731 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 739733 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 739734 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 739735 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 739735 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 739737 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 739737 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 739739 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 739739 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 739740 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 739741 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 739742 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 739742 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 739744 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 739744 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 739746 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 739746 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 739747 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 739748 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 740030 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_23AE55F0085E3EA5-001/control-001/cores/collection1
   [junit4]   2> 740032 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 740036 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@18525af5{/_uv/b,null,AVAILABLE}
   [junit4]   2> 740038 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.e.j.s.ServerConnector Started ServerConnector@6a2422e6{HTTP/1.1}{127.0.0.1:44328}
   [junit4]   2> 740038 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.e.j.s.Server Started @742688ms
   [junit4]   2> 740038 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_23AE55F0085E3EA5-001/tempDir-001/control/data, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_23AE55F0085E3EA5-001/control-001/cores, hostContext=/_uv/b, hostPort=44328}
   [junit4]   2> 740038 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@1fe91485
   [junit4]   2> 740039 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_23AE55F0085E3EA5-001/control-001/'
   [junit4]   2> 740039 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 740039 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 740055 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 740057 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 740057 INFO  (zkCallback-693-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@fc79d2a name:ZooKeeperConnection Watcher:127.0.0.1:48193/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 740057 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 740058 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 740059 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 740059 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_23AE55F0085E3EA5-001/control-001/solr.xml
   [junit4]   2> 740074 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_23AE55F0085E3EA5-001/control-001/cores
   [junit4]   2> 740074 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.CoreContainer New CoreContainer 44516556
   [junit4]   2> 740074 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_23AE55F0085E3EA5-001/control-001/]
   [junit4]   2> 740074 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_23AE55F0085E3EA5-001/control-001/lib
   [junit4]   2> 740074 WARN  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_23AE55F0085E3EA5-001/control-001/lib).
   [junit4]   2> 740083 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 740084 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 740084 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 740084 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 740084 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:48193/solr
   [junit4]   2> 740084 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 740085 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 740085 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 740087 INFO  (zkCallback-695-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@e4c277f name:ZooKeeperConnection Watcher:127.0.0.1:48193 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 740088 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 740088 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 740089 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [n:127.0.0.1:44328__uv%2Fb    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 740091 INFO  (zkCallback-696-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6b170d2a name:ZooKeeperConnection Watcher:127.0.0.1:48193/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 740091 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [n:127.0.0.1:44328__uv%2Fb    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 740092 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [n:127.0.0.1:44328__uv%2Fb    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 740093 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [n:127.0.0.1:44328__uv%2Fb    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 740107 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [n:127.0.0.1:44328__uv%2Fb    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 740109 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [n:127.0.0.1:44328__uv%2Fb    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 740111 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [n:127.0.0.1:44328__uv%2Fb    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 740112 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [n:127.0.0.1:44328__uv%2Fb    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 740114 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [n:127.0.0.1:44328__uv%2Fb    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 740115 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [n:127.0.0.1:44328__uv%2Fb    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 740116 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [n:127.0.0.1:44328__uv%2Fb    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 740117 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [n:127.0.0.1:44328__uv%2Fb    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:44328__uv%2Fb
   [junit4]   2> 740117 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [n:127.0.0.1:44328__uv%2Fb    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:44328__uv%2Fb
   [junit4]   2> 740119 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [n:127.0.0.1:44328__uv%2Fb    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 740120 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [n:127.0.0.1:44328__uv%2Fb    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 740121 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [n:127.0.0.1:44328__uv%2Fb    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 740121 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [n:127.0.0.1:44328__uv%2Fb    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94781574254166020-127.0.0.1:44328__uv%2Fb-n_0000000000
   [junit4]   2> 740122 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [n:127.0.0.1:44328__uv%2Fb    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:44328__uv%2Fb
   [junit4]   2> 740122 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [n:127.0.0.1:44328__uv%2Fb    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 740123 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [n:127.0.0.1:44328__uv%2Fb    ] o.a.s.c.Overseer Overseer (id=94781574254166020-127.0.0.1:44328__uv%2Fb-n_0000000000) starting
   [junit4]   2> 740124 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [n:127.0.0.1:44328__uv%2Fb    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 740130 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [n:127.0.0.1:44328__uv%2Fb    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 740130 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [n:127.0.0.1:44328__uv%2Fb    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 740130 INFO  (OverseerCollectionConfigSetProcessor-94781574254166020-127.0.0.1:44328__uv%2Fb-n_0000000000) [n:127.0.0.1:44328__uv%2Fb    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 740131 INFO  (OverseerStateUpdate-94781574254166020-127.0.0.1:44328__uv%2Fb-n_0000000000) [n:127.0.0.1:44328__uv%2Fb    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 740166 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [n:127.0.0.1:44328__uv%2Fb    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 740166 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [n:127.0.0.1:44328__uv%2Fb    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 740167 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [n:127.0.0.1:44328__uv%2Fb    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_23AE55F0085E3EA5-001/control-001/cores
   [junit4]   2> 740168 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [n:127.0.0.1:44328__uv%2Fb    ] o.a.s.c.SolrCore Created CoreDescriptor: {absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_23AE55F0085E3EA5-001/control-001/cores/collection1/, config=solrconfig.xml, name=collection1, dataDir=data/, transient=false, collection=control_collection, coreNodeName=, schema=schema.xml, configSetProperties=configsetprops.json, loadOnStartup=true, shard=, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_23AE55F0085E3EA5-001/control-001/cores/collection1}
   [junit4]   2> 740168 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [n:127.0.0.1:44328__uv%2Fb    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_23AE55F0085E3EA5-001/control-001/cores/collection1/
   [junit4]   2> 740168 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [n:127.0.0.1:44328__uv%2Fb    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 740169 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [n:127.0.0.1:44328__uv%2Fb    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 740169 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [n:127.0.0.1:44328__uv%2Fb    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 740169 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 740169 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 740171 INFO  (OverseerStateUpdate-94781574254166020-127.0.0.1:44328__uv%2Fb-n_0000000000) [n:127.0.0.1:44328__uv%2Fb    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "base_url":"http://127.0.0.1:44328/_uv/b",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "node_name":"127.0.0.1:44328__uv%2Fb",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core":"collection1"} current state version: 0
   [junit4]   2> 740172 INFO  (OverseerStateUpdate-94781574254166020-127.0.0.1:44328__uv%2Fb-n_0000000000) [n:127.0.0.1:44328__uv%2Fb    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "base_url":"http://127.0.0.1:44328/_uv/b",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "node_name":"127.0.0.1:44328__uv%2Fb",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core":"collection1"}
   [junit4]   2> 740172 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 740172 INFO  (OverseerStateUpdate-94781574254166020-127.0.0.1:44328__uv%2Fb-n_0000000000) [n:127.0.0.1:44328__uv%2Fb    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 740172 INFO  (OverseerStateUpdate-94781574254166020-127.0.0.1:44328__uv%2Fb-n_0000000000) [n:127.0.0.1:44328__uv%2Fb    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 740174 INFO  (zkCallback-696-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 741172 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 741172 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 741173 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 741173 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 741174 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 741174 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_23AE55F0085E3EA5-001/control-001/cores/collection1/'
   [junit4]   2> 741174 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 741174 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 741189 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 741195 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 741208 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 741236 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 741237 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 741243 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 741485 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 741494 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 741496 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 741509 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 741512 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 741515 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 741516 WARN  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 741516 WARN  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 741516 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 741516 WARN  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 741516 WARN  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 741517 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 741517 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 741517 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 741518 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_23AE55F0085E3EA5-001/control-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 741518 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@70455d96
   [junit4]   2> 741518 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_23AE55F0085E3EA5-001/control-001/cores/collection1/data
   [junit4]   2> 741518 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_23AE55F0085E3EA5-001/control-001/cores/collection1/data/index/
   [junit4]   2> 741519 WARN  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_23AE55F0085E3EA5-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 741519 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_23AE55F0085E3EA5-001/control-001/cores/collection1/data/index
   [junit4]   2> 741519 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=38, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.7776191985533545]
   [junit4]   2> 741519 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=RawDirectoryWrapper(RAMDirectory@255d1451 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1e0e501),segFN=segments_1,generation=1}
   [junit4]   2> 741519 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 741520 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_23AE55F0085E3EA5-001/control-001/cores/collection1/data/
   [junit4]   2> 741520 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_23AE55F0085E3EA5-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 741524 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 741525 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 741525 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 741525 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 741525 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 741525 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 741525 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 741526 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 741526 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 741528 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 741530 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 741531 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 741532 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 741536 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /replication,/schema,/admin/segments,/admin/ping,/admin/properties,/update,/admin/mbeans,/admin/luke,/admin/file,/config,/admin/system,/update/json,/get,/update/json/docs,/admin/plugins,/admin/threads,/admin/logging,standard,/update/csv
   [junit4]   2> 741537 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 741538 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 741538 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 741539 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 741539 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 741539 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=26, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=0.84765625, noCFSRatio=0.30815756924504567]
   [junit4]   2> 741539 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=RawDirectoryWrapper(RAMDirectory@255d1451 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1e0e501),segFN=segments_1,generation=1}
   [junit4]   2> 741539 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 741540 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@271429c8[collection1] main
   [junit4]   2> 741540 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 741541 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 741541 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 741541 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 741541 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 741541 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 741541 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 741542 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 741542 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 741542 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 741542 INFO  (searcherExecutor-1884-thread-1-processing-r:core_node1 s:shard1 c:control_collection n:127.0.0.1:44328__uv%2Fb x:collection1) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@271429c8[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 741542 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 741542 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 741542 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 741543 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1516505190091456512
   [junit4]   2> 741545 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 2.0ms to seed version buckets with highest version 1516505190091456512
   [junit4]   2> 741545 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 741545 INFO  (coreLoadExecutor-1883-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 741546 INFO  (coreZkRegister-1877-thread-1-processing-r:core_node1 s:shard1 c:control_collection n:127.0.0.1:44328__uv%2Fb x:collection1) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:44328/_uv/b collection:control_collection shard:shard1
   [junit4]   2> 741547 INFO  (coreZkRegister-1877-thread-1-processing-r:core_node1 s:shard1 c:control_collection n:127.0.0.1:44328__uv%2Fb x:collection1) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 741550 INFO  (coreZkRegister-1877-thread-1-processing-r:core_node1 s:shard1 c:control_collection n:127.0.0.1:44328__uv%2Fb x:collection1) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94781574254166020-core_node1-n_0000000000
   [junit4]   2> 741551 INFO  (coreZkRegister-1877-thread-1-processing-r:core_node1 s:shard1 c:control_collection n:127.0.0.1:44328__uv%2Fb x:collection1) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 741552 INFO  (coreZkRegister-1877-thread-1-processing-r:core_node1 s:shard1 c:control_collection n:127.0.0.1:44328__uv%2Fb x:collection1) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 741552 INFO  (coreZkRegister-1877-thread-1-processing-r:core_node1 s:shard1 c:control_collection n:127.0.0.1:44328__uv%2Fb x:collection1) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 741552 INFO  (coreZkRegister-1877-thread-1-processing-r:core_node1 s:shard1 c:control_collection n:127.0.0.1:44328__uv%2Fb x:collection1) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:44328/_uv/b/collection1/
   [junit4]   2> 741553 INFO  (coreZkRegister-1877-thread-1-processing-r:core_node1 s:shard1 c:control_collection n:127.0.0.1:44328__uv%2Fb x:collection1) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 741553 INFO  (OverseerStateUpdate-94781574254166020-127.0.0.1:44328__uv%2Fb-n_0000000000) [n:127.0.0.1:44328__uv%2Fb    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 741553 INFO  (coreZkRegister-1877-thread-1-processing-r:core_node1 s:shard1 c:control_collection n:127.0.0.1:44328__uv%2Fb x:collection1) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:44328/_uv/b/collection1/ has no replicas
   [junit4]   2> 741553 INFO  (coreZkRegister-1877-thread-1-processing-r:core_node1 s:shard1 c:control_collection n:127.0.0.1:44328__uv%2Fb x:collection1) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:44328/_uv/b/collection1/ shard1
   [junit4]   2> 741553 INFO  (coreZkRegister-1877-thread-1-processing-r:core_node1 s:shard1 c:control_collection n:127.0.0.1:44328__uv%2Fb x:collection1) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 741556 INFO  (coreZkRegister-1877-thread-1-processing-r:core_node1 s:shard1 c:control_collection n:127.0.0.1:44328__uv%2Fb x:collection1) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 741558 INFO  (OverseerStateUpdate-94781574254166020-127.0.0.1:44328__uv%2Fb-n_0000000000) [n:127.0.0.1:44328__uv%2Fb    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:44328/_uv/b",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 1
   [junit4]   2> 741660 INFO  (zkCallback-696-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 741671 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 741671 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 741673 INFO  (zkCallback-698-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@769927aa name:ZooKeeperConnection Watcher:127.0.0.1:48193/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 741673 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 741673 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 741673 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 741675 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 741709 INFO  (coreZkRegister-1877-thread-1-processing-r:core_node1 s:shard1 c:control_collection n:127.0.0.1:44328__uv%2Fb x:collection1) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:44328/_uv/b/collection1/ and leader is http://127.0.0.1:44328/_uv/b/collection1/
   [junit4]   2> 741709 INFO  (coreZkRegister-1877-thread-1-processing-r:core_node1 s:shard1 c:control_collection n:127.0.0.1:44328__uv%2Fb x:collection1) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:44328/_uv/b
   [junit4]   2> 741709 INFO  (coreZkRegister-1877-thread-1-processing-r:core_node1 s:shard1 c:control_collection n:127.0.0.1:44328__uv%2Fb x:collection1) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 741709 INFO  (coreZkRegister-1877-thread-1-processing-r:core_node1 s:shard1 c:control_collection n:127.0.0.1:44328__uv%2Fb x:collection1) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 741709 INFO  (coreZkRegister-1877-thread-1-processing-r:core_node1 s:shard1 c:control_collection n:127.0.0.1:44328__uv%2Fb x:collection1) [n:127.0.0.1:44328__uv%2Fb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 741711 INFO  (OverseerStateUpdate-94781574254166020-127.0.0.1:44328__uv%2Fb-n_0000000000) [n:127.0.0.1:44328__uv%2Fb    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "base_url":"http://127.0.0.1:44328/_uv/b",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "node_name":"127.0.0.1:44328__uv%2Fb",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core":"collection1"} current state version: 2
   [junit4]   2> 741712 INFO  (OverseerStateUpdate-94781574254166020-127.0.0.1:44328__uv%2Fb-n_0000000000) [n:127.0.0.1:44328__uv%2Fb    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "base_url":"http://127.0.0.1:44328/_uv/b",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "node_name":"127.0.0.1:44328__uv%2Fb",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core":"collection1"}
   [junit4]   2> 741814 INFO  (zkCallback-698-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 741814 INFO  (zkCallback-696-thread-1-processing-n:127.0.0.1:44328__uv%2Fb) [n:127.0.0.1:44328__uv%2Fb    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 741949 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_23AE55F0085E3EA5-001/shard-1-001/cores/collection1
   [junit4]   2> 741950 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_23AE55F0085E3EA5-001/shard-1-001
   [junit4]   2> 741951 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 741952 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@3232c196{/_uv/b,null,AVAILABLE}
   [junit4]   2> 741953 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.e.j.s.ServerConnector Started ServerConnector@7ba29661{HTTP/1.1}{127.0.0.1:58797}
   [junit4]   2> 741953 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.e.j.s.Server Started @744603ms
   [junit4]   2> 741953 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostPort=58797, hostContext=/_uv/b, solrconfig=solrconfig.xml, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_23AE55F0085E3EA5-001/tempDir-001/jetty1, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_23AE55F0085E3EA5-001/shard-1-001/cores}
   [junit4]   2> 741953 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@1fe91485
   [junit4]   2> 741953 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_23AE55F0085E3EA5-001/shard-1-001/'
   [junit4]   2> 741953 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 741953 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 741970 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 741970 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 741972 INFO  (zkCallback-699-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@eeef72c name:ZooKeeperConnection Watcher:127.0.0.1:48193/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 741972 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 741972 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 741973 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 741973 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_23AE55F0085E3EA5-001/shard-1-001/solr.xml
   [junit4]   2> 741987 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_23AE55F0085E3EA5-001/shard-1-001/cores
   [junit4]   2> 741987 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.CoreContainer New CoreContainer 274208664
   [junit4]   2> 741987 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_23AE55F0085E3EA5-001/shard-1-001/]
   [junit4]   2> 741987 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_23AE55F0085E3EA5-001/shard-1-001/lib
   [junit4]   2> 741988 WARN  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_23AE55F0085E3EA5-001/shard-1-001/lib).
   [junit4]   2> 741996 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 741997 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 741997 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 741997 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 741997 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:48193/solr
   [junit4]   2> 741997 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 741997 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 741998 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 741999 INFO  (zkCallback-701-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7fb00281 name:ZooKeeperConnection Watcher:127.0.0.1:48193 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 742000 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 742000 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 742002 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [n:127.0.0.1:58797__uv%2Fb    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 742003 INFO  (zkCallback-702-thread-1-processing-n:127.0.0.1:58797__uv%2Fb) [n:127.0.0.1:58797__uv%2Fb    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@314f2223 name:ZooKeeperConnection Watcher:127.0.0.1:48193/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 742004 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[23AE55F0085E3EA5]) [n:127.0

[...truncated too long message...]

        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]   2>         at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> Oct 31, 2015 2:21:58 AM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
   [junit4]   2> INFO: Starting to interrupt leaked threads:
   [junit4]   2>    1) Thread[id=18893, name=searcherExecutor-6813-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]   2> Oct 31, 2015 2:22:01 AM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
   [junit4]   2> SEVERE: There are still zombie threads that couldn't be terminated:
   [junit4]   2>    1) Thread[id=18893, name=searcherExecutor-6813-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]   2>         at sun.misc.Unsafe.park(Native Method)
   [junit4]   2>         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4]   2>         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
   [junit4]   2>         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]   2>         at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene54): {range_facet_l=PostingsFormat(name=LuceneVarGapFixedInterval), _version_=PostingsFormat(name=LuceneVarGapFixedInterval), range_facet_i_dv=PostingsFormat(name=LuceneVarGapFixedInterval), range_facet_l_dv=PostingsFormat(name=Memory doPackFST= true), timestamp=PostingsFormat(name=LuceneVarGapFixedInterval), intDefault=PostingsFormat(name=LuceneVarGapFixedInterval), multiDefault=PostingsFormat(name=Asserting), id=PostingsFormat(name=Memory doPackFST= true), other_tl1=PostingsFormat(name=LuceneVarGapFixedInterval), a_t=PostingsFormat(name=LuceneVarGapFixedInterval), text=BlockTreeOrds(blocksize=128), a_i1=PostingsFormat(name=LuceneVarGapFixedInterval)}, docValues:{range_facet_i_dv=DocValuesFormat(name=Lucene54), range_facet_l_dv=DocValuesFormat(name=Direct), timestamp=DocValuesFormat(name=Lucene54)}, sim=RandomSimilarityProvider(queryNorm=true,coord=no): {}, locale=es_DO, timezone=Asia/Hebron
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_72 (64-bit)/cpus=4,threads=2,free=73202784,total=477102080
   [junit4]   2> NOTE: All tests run in this JVM: [TestManagedResourceStorage, TestFieldTypeResource, TestCrossCoreJoin, CoreAdminRequestStatusTest, SpatialRPTFieldTypeTest, TestReloadDeadlock, SegmentsInfoRequestHandlerTest, RequestLoggingTest, TestCopyFieldCollectionResource, TestCoreContainer, TestUtils, TestStressVersions, TestOverriddenPrefixQueryForCustomFieldType, HdfsNNFailoverTest, ChaosMonkeyNothingIsSafeTest, HdfsThreadLeakTest, TestWordDelimiterFilterFactory, SpatialFilterTest, DistributedVersionInfoTest, TermVectorComponentTest, TestObjectReleaseTracker, TestJmxMonitoredMap, TestLFUCache, TestExactStatsCache, WordBreakSolrSpellCheckerTest, SuggestComponentContextFilterQueryTest, DistributedFacetPivotWhiteBoxTest, OverseerTaskQueueTest, SolrInfoMBeanTest, PKIAuthenticationIntegrationTest, PreAnalyzedUpdateProcessorTest, RollingRestartTest, TestFiltering, SpellCheckComponentTest, OverseerStatusTest, BufferStoreTest, TestHighlightDedupGrouping, ClusterStateUpdateTest, TestAddFieldRealTimeGet, SliceStateTest, TestTrieFacet, ChaosMonkeySafeLeaderTest, TestSchemaResource, TestSolrCoreProperties, TestBulkSchemaConcurrent, BasicFunctionalityTest, SaslZkACLProviderTest, SolrXmlInZkTest, SchemaVersionSpecificBehaviorTest, TestHashPartitioner, TestLeaderInitiatedRecoveryThread, SpellPossibilityIteratorTest, AtomicUpdatesTest, MBeansHandlerTest, TestLRUStatsCache, TestInfoStreamLogging, TestNoOpRegenerator, TestRuleBasedAuthorizationPlugin, TestManagedSchemaDynamicFieldResource, CollectionStateFormat2Test, DistanceUnitsTest, RankQueryTest, FastVectorHighlighterTest, DocValuesTest, DistributedFacetPivotSmallTest, CursorMarkTest, ShardRoutingCustomTest, CircularListTest, TestImplicitCoreProperties, TestLMJelinekMercerSimilarityFactory, ShardRoutingTest, DistanceFunctionTest, IndexBasedSpellCheckerTest, TestAnalyzeInfixSuggestions, SpellCheckCollatorTest, TestSuggestSpellingConverter, TestHdfsUpdateLog, LoggingHandlerTest, TestRTGBase, TestBlendedInfixSuggestions, SuggesterFSTTest, SecurityConfHandlerTest, SolrIndexSplitterTest, TermsComponentTest, TestBM25SimilarityFactory, NotRequiredUniqueKeyTest, ZkCLITest, TestSortingResponseWriter, BasicDistributedZk2Test, UnloadDistributedZkTest, CollectionsAPIDistributedZkTest]
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=CollectionsAPIDistributedZkTest -Dtests.seed=23AE55F0085E3EA5 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=es_DO -Dtests.timezone=Asia/Hebron -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR   0.00s J1 | CollectionsAPIDistributedZkTest (suite) <<<
   [junit4]    > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.cloud.CollectionsAPIDistributedZkTest: 
   [junit4]    >    1) Thread[id=18893, name=searcherExecutor-6813-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
   [junit4]    >         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([23AE55F0085E3EA5]:0)Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   [junit4]    >    1) Thread[id=18893, name=searcherExecutor-6813-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
   [junit4]    >         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([23AE55F0085E3EA5]:0)
   [junit4] Completed [543/543] on J1 in 4836.62s, 1 test, 3 errors <<< FAILURES!

[...truncated 1 lines...]
   [junit4] JVM J1: stdout was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J1-20151030_202826_187.sysout
   [junit4] >>> JVM J1: stdout (verbatim) ----
   [junit4] java.lang.OutOfMemoryError: Java heap space
   [junit4] Dumping heap to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/heapdumps/java_pid18125.hprof ...
   [junit4] Heap dump file created [630003199 bytes in 11.073 secs]
   [junit4] <<< JVM J1: EOF ----

[...truncated 13 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 543 suites (7 ignored), 2147 tests, 2 suite-level errors, 2 errors, 1 failure, 31 ignored (18 assumptions) [seed: 23AE55F0085E3EA5]

Total time: 246 minutes 58 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1001 - Still Failing

[Apache Jenkins Server <je...@builds.apache.org>]

Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1001/

4 tests failed.
FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
KeeperErrorCode = Session expired for /clusterstate.json

Stack Trace:
org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /clusterstate.json
	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
	at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
	at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:345)
	at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:342)
	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:61)
	at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:342)
	at org.apache.solr.common.cloud.ZkStateReader.refreshLegacyClusterState(ZkStateReader.java:477)
	at org.apache.solr.common.cloud.ZkStateReader.updateClusterState(ZkStateReader.java:257)
	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForCollectionToDisappear(AbstractDistribZkTestBase.java:196)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.assertCollectionNotExists(AbstractFullDistribZkTestBase.java:1772)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.deletePartiallyCreatedCollection(CollectionsAPIDistributedZkTest.java:234)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test(CollectionsAPIDistributedZkTest.java:171)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1660)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:866)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:902)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:916)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:963)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:938)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:875)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:777)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:811)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:822)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.CollectionsAPIDistributedZkTest

Error Message:
2 threads leaked from SUITE scope at org.apache.solr.cloud.CollectionsAPIDistributedZkTest:     1) Thread[id=195553, name=searcherExecutor-2346-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)    2) Thread[id=195585, name=searcherExecutor-2359-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 2 threads leaked from SUITE scope at org.apache.solr.cloud.CollectionsAPIDistributedZkTest: 
   1) Thread[id=195553, name=searcherExecutor-2346-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
   2) Thread[id=195585, name=searcherExecutor-2359-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
	at __randomizedtesting.SeedInfo.seed([6263BEB3B251C403]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.CollectionsAPIDistributedZkTest

Error Message:
There are still zombie threads that couldn't be terminated:    1) Thread[id=195585, name=searcherExecutor-2359-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)    2) Thread[id=195553, name=searcherExecutor-2346-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   1) Thread[id=195585, name=searcherExecutor-2359-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
   2) Thread[id=195553, name=searcherExecutor-2346-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
	at __randomizedtesting.SeedInfo.seed([6263BEB3B251C403]:0)


FAILED:  org.apache.solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR

Error Message:
Captured an uncaught exception in thread: Thread[id=606, name=coreZkRegister-205-thread-1, state=RUNNABLE, group=TGRP-LeaderInitiatedRecoveryOnShardRestartTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=606, name=coreZkRegister-205-thread-1, state=RUNNABLE, group=TGRP-LeaderInitiatedRecoveryOnShardRestartTest]
Caused by: java.lang.AssertionError
	at __randomizedtesting.SeedInfo.seed([6263BEB3B251C403]:0)
	at org.apache.solr.cloud.ZkController.updateLeaderInitiatedRecoveryState(ZkController.java:2133)
	at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:434)
	at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)
	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)
	at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:346)
	at org.apache.solr.cloud.ZkController.joinElection(ZkController.java:1113)
	at org.apache.solr.cloud.ZkController.register(ZkController.java:926)
	at org.apache.solr.cloud.ZkController.register(ZkController.java:881)
	at org.apache.solr.core.ZkContainer$2.run(ZkContainer.java:183)
	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 10206 lines...]
   [junit4] Suite: org.apache.solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_6263BEB3B251C403-001/init-core-data-001
   [junit4]   2> 111819 INFO  (SUITE-LeaderInitiatedRecoveryOnShardRestartTest-seed#[6263BEB3B251C403]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /ktdm/
   [junit4]   2> 111824 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 111824 INFO  (Thread-155) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 111824 INFO  (Thread-155) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 111924 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.ZkTestServer start zk server on port:33745
   [junit4]   2> 111924 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 111925 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 111928 INFO  (zkCallback-55-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@3959b9e2 name:ZooKeeperConnection Watcher:127.0.0.1:33745 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 111928 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 111928 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 111928 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 111931 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 111932 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 111934 INFO  (zkCallback-56-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@4b615046 name:ZooKeeperConnection Watcher:127.0.0.1:33745/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 111934 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 111934 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 111934 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 111937 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 111940 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 111943 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 111946 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 111946 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 111951 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 111951 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 111954 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 111954 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 111957 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 111958 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 111962 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 111962 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 111965 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 111965 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 111968 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 111969 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 111972 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 111972 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 111975 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 111975 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 111979 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 111980 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 111984 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 111985 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 112389 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_6263BEB3B251C403-001/control-001/cores/collection1
   [junit4]   2> 112391 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 112396 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@7f5fd0{/ktdm,null,AVAILABLE}
   [junit4]   2> 112407 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.e.j.s.ServerConnector Started ServerConnector@6b8f13f8{HTTP/1.1}{127.0.0.1:40092}
   [junit4]   2> 112413 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.e.j.s.Server Started @115143ms
   [junit4]   2> 112413 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_6263BEB3B251C403-001/tempDir-001/control/data, hostPort=40092, hostContext=/ktdm, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_6263BEB3B251C403-001/control-001/cores}
   [junit4]   2> 112414 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@1fe91485
   [junit4]   2> 112414 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_6263BEB3B251C403-001/control-001/'
   [junit4]   2> 112414 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 112414 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 112432 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 112434 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 112437 INFO  (zkCallback-57-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@18d14949 name:ZooKeeperConnection Watcher:127.0.0.1:33745/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 112437 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 112438 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 112441 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 112441 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_6263BEB3B251C403-001/control-001/solr.xml
   [junit4]   2> 112462 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_6263BEB3B251C403-001/control-001/cores
   [junit4]   2> 112462 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.CoreContainer New CoreContainer 1856592448
   [junit4]   2> 112462 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_6263BEB3B251C403-001/control-001/]
   [junit4]   2> 112462 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_6263BEB3B251C403-001/control-001/lib
   [junit4]   2> 112463 WARN  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_6263BEB3B251C403-001/control-001/lib).
   [junit4]   2> 112472 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 112473 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 112473 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 112473 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 112473 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:33745/solr
   [junit4]   2> 112473 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 112474 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 112475 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 112478 INFO  (zkCallback-59-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@34980b87 name:ZooKeeperConnection Watcher:127.0.0.1:33745 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 112479 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 112480 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 112486 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [n:127.0.0.1:40092_ktdm    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 112489 INFO  (zkCallback-60-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6c9c1bb2 name:ZooKeeperConnection Watcher:127.0.0.1:33745/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 112489 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [n:127.0.0.1:40092_ktdm    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 112493 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [n:127.0.0.1:40092_ktdm    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 112498 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [n:127.0.0.1:40092_ktdm    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 112503 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [n:127.0.0.1:40092_ktdm    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 112511 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [n:127.0.0.1:40092_ktdm    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 112517 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [n:127.0.0.1:40092_ktdm    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 112524 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [n:127.0.0.1:40092_ktdm    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 112528 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [n:127.0.0.1:40092_ktdm    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 112530 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [n:127.0.0.1:40092_ktdm    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 112532 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [n:127.0.0.1:40092_ktdm    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 112537 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [n:127.0.0.1:40092_ktdm    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:40092_ktdm
   [junit4]   2> 112537 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [n:127.0.0.1:40092_ktdm    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:40092_ktdm
   [junit4]   2> 112541 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [n:127.0.0.1:40092_ktdm    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 112544 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [n:127.0.0.1:40092_ktdm    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 112548 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [n:127.0.0.1:40092_ktdm    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 112549 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [n:127.0.0.1:40092_ktdm    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94775665174380548-127.0.0.1:40092_ktdm-n_0000000000
   [junit4]   2> 112550 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [n:127.0.0.1:40092_ktdm    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:40092_ktdm
   [junit4]   2> 112550 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [n:127.0.0.1:40092_ktdm    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 112552 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [n:127.0.0.1:40092_ktdm    ] o.a.s.c.Overseer Overseer (id=94775665174380548-127.0.0.1:40092_ktdm-n_0000000000) starting
   [junit4]   2> 112556 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [n:127.0.0.1:40092_ktdm    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 112570 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [n:127.0.0.1:40092_ktdm    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 112571 INFO  (OverseerStateUpdate-94775665174380548-127.0.0.1:40092_ktdm-n_0000000000) [n:127.0.0.1:40092_ktdm    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 112571 INFO  (OverseerCollectionConfigSetProcessor-94775665174380548-127.0.0.1:40092_ktdm-n_0000000000) [n:127.0.0.1:40092_ktdm    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 112572 INFO  (OverseerStateUpdate-94775665174380548-127.0.0.1:40092_ktdm-n_0000000000) [n:127.0.0.1:40092_ktdm    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 112583 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [n:127.0.0.1:40092_ktdm    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 112610 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [n:127.0.0.1:40092_ktdm    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 112610 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [n:127.0.0.1:40092_ktdm    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 112611 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [n:127.0.0.1:40092_ktdm    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_6263BEB3B251C403-001/control-001/cores
   [junit4]   2> 112612 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [n:127.0.0.1:40092_ktdm    ] o.a.s.c.SolrCore Created CoreDescriptor: {dataDir=data/, coreNodeName=, config=solrconfig.xml, transient=false, collection=control_collection, schema=schema.xml, loadOnStartup=true, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_6263BEB3B251C403-001/control-001/cores/collection1/, configSetProperties=configsetprops.json, shard=, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_6263BEB3B251C403-001/control-001/cores/collection1, name=collection1}
   [junit4]   2> 112612 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [n:127.0.0.1:40092_ktdm    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_6263BEB3B251C403-001/control-001/cores/collection1/
   [junit4]   2> 112612 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [n:127.0.0.1:40092_ktdm    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 112613 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [n:127.0.0.1:40092_ktdm    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 112613 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [n:127.0.0.1:40092_ktdm    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 112614 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 112614 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 112621 INFO  (OverseerStateUpdate-94775665174380548-127.0.0.1:40092_ktdm-n_0000000000) [n:127.0.0.1:40092_ktdm    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "roles":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:40092/ktdm",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "node_name":"127.0.0.1:40092_ktdm",
   [junit4]   2>   "core":"collection1"} current state version: 0
   [junit4]   2> 112625 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 112632 INFO  (OverseerStateUpdate-94775665174380548-127.0.0.1:40092_ktdm-n_0000000000) [n:127.0.0.1:40092_ktdm    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "roles":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:40092/ktdm",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "node_name":"127.0.0.1:40092_ktdm",
   [junit4]   2>   "core":"collection1"}
   [junit4]   2> 112633 INFO  (OverseerStateUpdate-94775665174380548-127.0.0.1:40092_ktdm-n_0000000000) [n:127.0.0.1:40092_ktdm    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 112634 INFO  (OverseerStateUpdate-94775665174380548-127.0.0.1:40092_ktdm-n_0000000000) [n:127.0.0.1:40092_ktdm    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 112646 INFO  (zkCallback-60-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 112653 INFO  (zkCallback-60-thread-2-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 113634 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 113634 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 113635 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 113635 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 113636 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 113636 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_6263BEB3B251C403-001/control-001/cores/collection1/'
   [junit4]   2> 113636 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 113636 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 113654 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 113672 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 113707 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 113742 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 113744 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 113787 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 114168 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 114180 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 114182 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 114220 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 114238 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 114241 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 114242 WARN  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 114242 WARN  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 114242 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 114243 WARN  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 114244 WARN  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 114245 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 114245 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 114245 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 114245 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_6263BEB3B251C403-001/control-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 114245 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@36036784
   [junit4]   2> 114247 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_6263BEB3B251C403-001/control-001/cores/collection1/data
   [junit4]   2> 114247 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_6263BEB3B251C403-001/control-001/cores/collection1/data/index/
   [junit4]   2> 114247 WARN  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_6263BEB3B251C403-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 114247 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_6263BEB3B251C403-001/control-001/cores/collection1/data/index
   [junit4]   2> 114248 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=42, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.12810220323990962]
   [junit4]   2> 114248 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@63195110 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@13192444),segFN=segments_1,generation=1}
   [junit4]   2> 114248 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 114266 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_6263BEB3B251C403-001/control-001/cores/collection1/data/
   [junit4]   2> 114266 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_6263BEB3B251C403-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 114295 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 114309 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 114309 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 114309 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 114309 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 114309 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 114310 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 114310 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 114310 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 114313 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 114314 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 114316 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 114318 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 114329 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /config,/admin/ping,/admin/threads,/update/json/docs,standard,/admin/plugins,/admin/system,/admin/mbeans,/get,/admin/file,/schema,/update/json,/admin/segments,/update,/admin/luke,/admin/logging,/replication,/admin/properties,/update/csv
   [junit4]   2> 114333 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 114334 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 114334 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 114335 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 114335 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 114335 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@29955a9c
   [junit4]   2> 114337 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@63195110 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@13192444),segFN=segments_1,generation=1}
   [junit4]   2> 114337 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 114337 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@4e1eabcc[collection1] main
   [junit4]   2> 114337 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 114338 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 114339 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 114339 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 114339 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 114339 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 114340 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 114340 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 114340 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 114340 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 114340 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 114341 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 114341 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 114341 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1516410645425160192
   [junit4]   2> 114343 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 2.0ms to seed version buckets with highest version 1516410645425160192
   [junit4]   2> 114343 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 114344 INFO  (coreLoadExecutor-167-thread-1-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 114346 INFO  (searcherExecutor-168-thread-1-processing-r:core_node1 c:control_collection s:shard1 n:127.0.0.1:40092_ktdm x:collection1) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@4e1eabcc[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 114354 INFO  (coreZkRegister-161-thread-1-processing-r:core_node1 c:control_collection s:shard1 n:127.0.0.1:40092_ktdm x:collection1) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:40092/ktdm collection:control_collection shard:shard1
   [junit4]   2> 114356 INFO  (coreZkRegister-161-thread-1-processing-r:core_node1 c:control_collection s:shard1 n:127.0.0.1:40092_ktdm x:collection1) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 114370 INFO  (coreZkRegister-161-thread-1-processing-r:core_node1 c:control_collection s:shard1 n:127.0.0.1:40092_ktdm x:collection1) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94775665174380548-core_node1-n_0000000000
   [junit4]   2> 114371 INFO  (coreZkRegister-161-thread-1-processing-r:core_node1 c:control_collection s:shard1 n:127.0.0.1:40092_ktdm x:collection1) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 114377 INFO  (coreZkRegister-161-thread-1-processing-r:core_node1 c:control_collection s:shard1 n:127.0.0.1:40092_ktdm x:collection1) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 114377 INFO  (coreZkRegister-161-thread-1-processing-r:core_node1 c:control_collection s:shard1 n:127.0.0.1:40092_ktdm x:collection1) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 114377 INFO  (coreZkRegister-161-thread-1-processing-r:core_node1 c:control_collection s:shard1 n:127.0.0.1:40092_ktdm x:collection1) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:40092/ktdm/collection1/
   [junit4]   2> 114377 INFO  (OverseerStateUpdate-94775665174380548-127.0.0.1:40092_ktdm-n_0000000000) [n:127.0.0.1:40092_ktdm    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 114377 INFO  (coreZkRegister-161-thread-1-processing-r:core_node1 c:control_collection s:shard1 n:127.0.0.1:40092_ktdm x:collection1) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 114377 INFO  (coreZkRegister-161-thread-1-processing-r:core_node1 c:control_collection s:shard1 n:127.0.0.1:40092_ktdm x:collection1) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:40092/ktdm/collection1/ has no replicas
   [junit4]   2> 114377 INFO  (coreZkRegister-161-thread-1-processing-r:core_node1 c:control_collection s:shard1 n:127.0.0.1:40092_ktdm x:collection1) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:40092/ktdm/collection1/ shard1
   [junit4]   2> 114378 INFO  (coreZkRegister-161-thread-1-processing-r:core_node1 c:control_collection s:shard1 n:127.0.0.1:40092_ktdm x:collection1) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 114378 INFO  (zkCallback-60-thread-2-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 114379 INFO  (zkCallback-60-thread-3-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 114388 INFO  (coreZkRegister-161-thread-1-processing-r:core_node1 c:control_collection s:shard1 n:127.0.0.1:40092_ktdm x:collection1) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 114394 INFO  (OverseerStateUpdate-94775665174380548-127.0.0.1:40092_ktdm-n_0000000000) [n:127.0.0.1:40092_ktdm    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:40092/ktdm",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 2
   [junit4]   2> 114499 INFO  (zkCallback-60-thread-3-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 114499 INFO  (zkCallback-60-thread-2-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 114546 INFO  (coreZkRegister-161-thread-1-processing-r:core_node1 c:control_collection s:shard1 n:127.0.0.1:40092_ktdm x:collection1) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:40092/ktdm/collection1/ and leader is http://127.0.0.1:40092/ktdm/collection1/
   [junit4]   2> 114546 INFO  (coreZkRegister-161-thread-1-processing-r:core_node1 c:control_collection s:shard1 n:127.0.0.1:40092_ktdm x:collection1) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:40092/ktdm
   [junit4]   2> 114546 INFO  (coreZkRegister-161-thread-1-processing-r:core_node1 c:control_collection s:shard1 n:127.0.0.1:40092_ktdm x:collection1) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 114546 INFO  (coreZkRegister-161-thread-1-processing-r:core_node1 c:control_collection s:shard1 n:127.0.0.1:40092_ktdm x:collection1) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 114547 INFO  (coreZkRegister-161-thread-1-processing-r:core_node1 c:control_collection s:shard1 n:127.0.0.1:40092_ktdm x:collection1) [n:127.0.0.1:40092_ktdm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 114549 INFO  (OverseerStateUpdate-94775665174380548-127.0.0.1:40092_ktdm-n_0000000000) [n:127.0.0.1:40092_ktdm    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:40092/ktdm",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "node_name":"127.0.0.1:40092_ktdm",
   [junit4]   2>   "core":"collection1"} current state version: 3
   [junit4]   2> 114550 INFO  (OverseerStateUpdate-94775665174380548-127.0.0.1:40092_ktdm-n_0000000000) [n:127.0.0.1:40092_ktdm    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:40092/ktdm",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "node_name":"127.0.0.1:40092_ktdm",
   [junit4]   2>   "core":"collection1"}
   [junit4]   2> 114615 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 114617 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 114619 INFO  (zkCallback-62-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1cc85d01 name:ZooKeeperConnection Watcher:127.0.0.1:33745/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 114619 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 114620 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 114620 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 114624 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 114653 INFO  (zkCallback-60-thread-2-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 114653 INFO  (zkCallback-62-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 114654 INFO  (zkCallback-60-thread-3-processing-n:127.0.0.1:40092_ktdm) [n:127.0.0.1:40092_ktdm    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 114931 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_6263BEB3B251C403-001/shard-1-001/cores/collection1
   [junit4]   2> 114931 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_6263BEB3B251C403-001/shard-1-001
   [junit4]   2> 114933 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 114937 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@2fc16b7{/ktdm,null,AVAILABLE}
   [junit4]   2> 114937 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.e.j.s.ServerConnector Started ServerConnector@743749e8{HTTP/1.1}{127.0.0.1:43568}
   [junit4]   2> 114937 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.e.j.s.Server Started @117667ms
   [junit4]   2> 114937 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_6263BEB3B251C403-001/shard-1-001/cores, hostContext=/ktdm, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_6263BEB3B251C403-001/tempDir-001/jetty1, solrconfig=solrconfig.xml, hostPort=43568}
   [junit4]   2> 114938 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@1fe91485
   [junit4]   2> 114938 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_6263BEB3B251C403-001/shard-1-001/'
   [junit4]   2> 114938 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 114938 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 114956 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 114957 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 114959 INFO  (zkCallback-63-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5d14d58b name:ZooKeeperConnection Watcher:127.0.0.1:33745/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 114959 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 114960 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 114961 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 114961 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_6263BEB3B251C403-001/shard-1-001/solr.xml
   [junit4]   2> 114978 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_6263BEB3B251C403-001/shard-1-001/cores
   [junit4]   2> 114979 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.CoreContainer New CoreContainer 1964988894
   [junit4]   2> 114979 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_6263BEB3B251C403-001/shard-1-001/]
   [junit4]   2> 114979 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_6263BEB3B251C403-001/shard-1-001/lib
   [junit4]   2> 114979 WARN  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_6263BEB3B251C403-001/shard-1-001/lib).
   [junit4]   2> 115005 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 115006 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 115006 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 115007 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 115007 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:33745/solr
   [junit4]   2> 115007 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 115007 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 115009 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 115010 INFO  (zkCallback-65-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5a706271 name:ZooKeeperConnection Watcher:127.0.0.1:33745 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 115011 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 115012 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 115018 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [n:127.0.0.1:43568_ktdm    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 115021 INFO  (zkCallback-66-thread-1-processing-n:127.0.0.1:43568_ktdm) [n:127.0.0.1:43568_ktdm    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1e1eb68f name:ZooKeeperConnection Watcher:127.0.0.1:33745/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 115021 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [n:127.0.0.1:43568_ktdm    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 115030 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[6263BEB3B251C403]) [n:127.

[...truncated too long message...]

    1) Thread[id=195553, name=searcherExecutor-2346-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]   2>    2) Thread[id=195585, name=searcherExecutor-2359-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]   2> Oct 29, 2015 8:16:05 PM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
   [junit4]   2> SEVERE: There are still zombie threads that couldn't be terminated:
   [junit4]   2>    1) Thread[id=195585, name=searcherExecutor-2359-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]   2>         at sun.misc.Unsafe.park(Native Method)
   [junit4]   2>         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4]   2>         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
   [junit4]   2>         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]   2>         at java.lang.Thread.run(Thread.java:745)
   [junit4]   2>    2) Thread[id=195553, name=searcherExecutor-2346-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]   2>         at sun.misc.Unsafe.park(Native Method)
   [junit4]   2>         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4]   2>         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
   [junit4]   2>         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]   2>         at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> NOTE: test params are: codec=CheapBastard, sim=RandomSimilarityProvider(queryNorm=false,coord=crazy): {}, locale=es_GT, timezone=Asia/Istanbul
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_72 (64-bit)/cpus=4,threads=3,free=124637208,total=477102080
   [junit4]   2> NOTE: All tests run in this JVM: [TestPartialUpdateDeduplication, SolrInfoMBeanTest, BlockCacheTest, SegmentsInfoRequestHandlerTest, BufferStoreTest, TestSolrDynamicMBean, TestExactStatsCache, SimpleMLTQParserTest, TestBM25SimilarityFactory, NotRequiredUniqueKeyTest, TestLeaderInitiatedRecoveryThread, DocumentBuilderTest, OpenCloseCoreStressTest, ZkCLITest, TestImplicitCoreProperties, SortByFunctionTest, SuggesterFSTTest, IgnoreCommitOptimizeUpdateProcessorFactoryTest, DistributedFacetPivotSmallTest, HdfsBasicDistributedZkTest, UpdateRequestProcessorFactoryTest, AddSchemaFieldsUpdateProcessorFactoryTest, TestAnalyzeInfixSuggestions, CollectionsAPIDistributedZkTest]
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=CollectionsAPIDistributedZkTest -Dtests.seed=6263BEB3B251C403 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=es_GT -Dtests.timezone=Asia/Istanbul -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR   0.00s J1 | CollectionsAPIDistributedZkTest (suite) <<<
   [junit4]    > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 2 threads leaked from SUITE scope at org.apache.solr.cloud.CollectionsAPIDistributedZkTest: 
   [junit4]    >    1) Thread[id=195553, name=searcherExecutor-2346-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
   [junit4]    >         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >    2) Thread[id=195585, name=searcherExecutor-2359-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
   [junit4]    >         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([6263BEB3B251C403]:0)Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   [junit4]    >    1) Thread[id=195585, name=searcherExecutor-2359-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
   [junit4]    >         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >    2) Thread[id=195553, name=searcherExecutor-2346-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
   [junit4]    >         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([6263BEB3B251C403]:0)
   [junit4] Completed [216/543] on J1 in 711.72s, 1 test, 3 errors <<< FAILURES!

[...truncated 972 lines...]
   [junit4] JVM J1: stdout was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J1-20151029_193608_466.sysout
   [junit4] >>> JVM J1: stdout (verbatim) ----
   [junit4] java.lang.OutOfMemoryError: GC overhead limit exceeded
   [junit4] Dumping heap to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/heapdumps/java_pid26869.hprof ...
   [junit4] Heap dump file created [605716282 bytes in 5.090 secs]
   [junit4] <<< JVM J1: EOF ----

[...truncated 18 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 543 suites (7 ignored), 2147 tests, 2 suite-level errors, 2 errors, 1139 ignored (15 assumptions) [seed: 6263BEB3B251C403]

Total time: 166 minutes 17 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1000 - Still Failing

[Apache Jenkins Server <je...@builds.apache.org>]

Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1000/

1 tests failed.
FAILED:  org.apache.solr.cloud.TestReplicaProperties.test

Error Message:
Collection testcollection does not have roles evenly distributed. Collection is: DocCollection(testcollection)={   "shards":{     "shard1":{       "range":"80000000-b332ffff",       "state":"active",       "replicas":{         "core_node7":{           "base_url":"http://127.0.0.1:51380",           "node_name":"127.0.0.1:51380_",           "core":"testcollection_shard1_replica2",           "state":"active",           "leader":"true"},         "core_node9":{           "base_url":"http://127.0.0.1:45841",           "node_name":"127.0.0.1:45841_",           "core":"testcollection_shard1_replica1",           "state":"active",           "property.preferredleader":"true"},         "core_node10":{           "base_url":"http://127.0.0.1:57326",           "node_name":"127.0.0.1:57326_",           "core":"testcollection_shard1_replica3",           "state":"active"}}},     "shard2":{       "range":"b3330000-e665ffff",       "state":"active",       "replicas":{         "core_node1":{           "base_url":"http://127.0.0.1:42012",           "node_name":"127.0.0.1:42012_",           "core":"testcollection_shard2_replica1",           "state":"active"},         "core_node8":{           "base_url":"http://127.0.0.1:45841",           "node_name":"127.0.0.1:45841_",           "core":"testcollection_shard2_replica3",           "state":"active",           "leader":"true"},         "core_node15":{           "base_url":"http://127.0.0.1:34598",           "node_name":"127.0.0.1:34598_",           "core":"testcollection_shard2_replica2",           "state":"active",           "property.preferredleader":"true"}}},     "shard3":{       "range":"e6660000-1998ffff",       "state":"active",       "replicas":{         "core_node2":{           "base_url":"http://127.0.0.1:57326",           "node_name":"127.0.0.1:57326_",           "core":"testcollection_shard3_replica2",           "state":"active"},         "core_node5":{           "base_url":"http://127.0.0.1:51380",           "node_name":"127.0.0.1:51380_",           "core":"testcollection_shard3_replica1",           "state":"active",           "leader":"true",           "property.preferredleader":"true"},         "core_node14":{           "base_url":"http://127.0.0.1:42012",           "node_name":"127.0.0.1:42012_",           "core":"testcollection_shard3_replica3",           "state":"active"}}},     "shard4":{       "range":"19990000-4ccbffff",       "state":"active",       "replicas":{         "core_node4":{           "base_url":"http://127.0.0.1:51380",           "node_name":"127.0.0.1:51380_",           "core":"testcollection_shard4_replica3",           "state":"active",           "leader":"true"},         "core_node11":{           "base_url":"http://127.0.0.1:45841",           "node_name":"127.0.0.1:45841_",           "core":"testcollection_shard4_replica2",           "state":"active",           "property.preferredleader":"true"},         "core_node12":{           "base_url":"http://127.0.0.1:34598",           "node_name":"127.0.0.1:34598_",           "core":"testcollection_shard4_replica1",           "state":"active"}}},     "shard5":{       "range":"4ccc0000-7fffffff",       "state":"active",       "replicas":{         "core_node3":{           "base_url":"http://127.0.0.1:42012",           "node_name":"127.0.0.1:42012_",           "core":"testcollection_shard5_replica2",           "state":"active",           "leader":"true"},         "core_node6":{           "base_url":"http://127.0.0.1:57326",           "node_name":"127.0.0.1:57326_",           "core":"testcollection_shard5_replica1",           "state":"active",           "property.preferredleader":"true"},         "core_node13":{           "base_url":"http://127.0.0.1:34598",           "node_name":"127.0.0.1:34598_",           "core":"testcollection_shard5_replica3",           "state":"active"}}}},   "replicationFactor":"3",   "autoAddReplicas":"false",   "maxShardsPerNode":"16",   "router":{"name":"compositeId"}}

Stack Trace:
java.lang.AssertionError: Collection testcollection does not have roles evenly distributed. Collection is: DocCollection(testcollection)={
  "shards":{
    "shard1":{
      "range":"80000000-b332ffff",
      "state":"active",
      "replicas":{
        "core_node7":{
          "base_url":"http://127.0.0.1:51380",
          "node_name":"127.0.0.1:51380_",
          "core":"testcollection_shard1_replica2",
          "state":"active",
          "leader":"true"},
        "core_node9":{
          "base_url":"http://127.0.0.1:45841",
          "node_name":"127.0.0.1:45841_",
          "core":"testcollection_shard1_replica1",
          "state":"active",
          "property.preferredleader":"true"},
        "core_node10":{
          "base_url":"http://127.0.0.1:57326",
          "node_name":"127.0.0.1:57326_",
          "core":"testcollection_shard1_replica3",
          "state":"active"}}},
    "shard2":{
      "range":"b3330000-e665ffff",
      "state":"active",
      "replicas":{
        "core_node1":{
          "base_url":"http://127.0.0.1:42012",
          "node_name":"127.0.0.1:42012_",
          "core":"testcollection_shard2_replica1",
          "state":"active"},
        "core_node8":{
          "base_url":"http://127.0.0.1:45841",
          "node_name":"127.0.0.1:45841_",
          "core":"testcollection_shard2_replica3",
          "state":"active",
          "leader":"true"},
        "core_node15":{
          "base_url":"http://127.0.0.1:34598",
          "node_name":"127.0.0.1:34598_",
          "core":"testcollection_shard2_replica2",
          "state":"active",
          "property.preferredleader":"true"}}},
    "shard3":{
      "range":"e6660000-1998ffff",
      "state":"active",
      "replicas":{
        "core_node2":{
          "base_url":"http://127.0.0.1:57326",
          "node_name":"127.0.0.1:57326_",
          "core":"testcollection_shard3_replica2",
          "state":"active"},
        "core_node5":{
          "base_url":"http://127.0.0.1:51380",
          "node_name":"127.0.0.1:51380_",
          "core":"testcollection_shard3_replica1",
          "state":"active",
          "leader":"true",
          "property.preferredleader":"true"},
        "core_node14":{
          "base_url":"http://127.0.0.1:42012",
          "node_name":"127.0.0.1:42012_",
          "core":"testcollection_shard3_replica3",
          "state":"active"}}},
    "shard4":{
      "range":"19990000-4ccbffff",
      "state":"active",
      "replicas":{
        "core_node4":{
          "base_url":"http://127.0.0.1:51380",
          "node_name":"127.0.0.1:51380_",
          "core":"testcollection_shard4_replica3",
          "state":"active",
          "leader":"true"},
        "core_node11":{
          "base_url":"http://127.0.0.1:45841",
          "node_name":"127.0.0.1:45841_",
          "core":"testcollection_shard4_replica2",
          "state":"active",
          "property.preferredleader":"true"},
        "core_node12":{
          "base_url":"http://127.0.0.1:34598",
          "node_name":"127.0.0.1:34598_",
          "core":"testcollection_shard4_replica1",
          "state":"active"}}},
    "shard5":{
      "range":"4ccc0000-7fffffff",
      "state":"active",
      "replicas":{
        "core_node3":{
          "base_url":"http://127.0.0.1:42012",
          "node_name":"127.0.0.1:42012_",
          "core":"testcollection_shard5_replica2",
          "state":"active",
          "leader":"true"},
        "core_node6":{
          "base_url":"http://127.0.0.1:57326",
          "node_name":"127.0.0.1:57326_",
          "core":"testcollection_shard5_replica1",
          "state":"active",
          "property.preferredleader":"true"},
        "core_node13":{
          "base_url":"http://127.0.0.1:34598",
          "node_name":"127.0.0.1:34598_",
          "core":"testcollection_shard5_replica3",
          "state":"active"}}}},
  "replicationFactor":"3",
  "autoAddReplicas":"false",
  "maxShardsPerNode":"16",
  "router":{"name":"compositeId"}}
	at __randomizedtesting.SeedInfo.seed([D08670B1FAA54301:58D24F6B54592EF9]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.solr.cloud.ReplicaPropertiesBase.verifyUnique(ReplicaPropertiesBase.java:176)
	at org.apache.solr.cloud.ReplicaPropertiesBase.verifyUniqueAcrossCollection(ReplicaPropertiesBase.java:107)
	at org.apache.solr.cloud.TestReplicaProperties.clusterAssignPropertyTest(TestReplicaProperties.java:108)
	at org.apache.solr.cloud.TestReplicaProperties.test(TestReplicaProperties.java:70)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1665)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:864)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:900)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:914)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:963)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:938)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:873)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:775)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:809)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:820)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 11383 lines...]
   [junit4] Suite: org.apache.solr.cloud.TestReplicaProperties
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestReplicaProperties_D08670B1FAA54301-001/init-core-data-001
   [junit4]   2> 3277076 WARN  (SUITE-TestReplicaProperties-seed#[D08670B1FAA54301]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=22 numCloses=22
   [junit4]   2> 3277076 INFO  (SUITE-TestReplicaProperties-seed#[D08670B1FAA54301]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false)
   [junit4]   2> 3277076 INFO  (SUITE-TestReplicaProperties-seed#[D08670B1FAA54301]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   2> 3277081 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 3277081 INFO  (Thread-6689) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 3277081 INFO  (Thread-6689) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 3277181 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.ZkTestServer start zk server on port:47499
   [junit4]   2> 3277181 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3277182 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3277185 INFO  (zkCallback-2471-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1bdb49b name:ZooKeeperConnection Watcher:127.0.0.1:47499 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3277185 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3277185 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3277185 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 3277188 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3277188 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3277189 INFO  (zkCallback-2472-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6f757b19 name:ZooKeeperConnection Watcher:127.0.0.1:47499/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3277189 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3277190 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3277190 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 3277192 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 3277193 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 3277195 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 3277196 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 3277196 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 3277198 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
   [junit4]   2> 3277199 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 3277200 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 3277201 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 3277202 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 3277202 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 3277204 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 3277204 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 3277205 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 3277205 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 3277207 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 3277207 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 3277209 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 3277209 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 3277210 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 3277210 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 3277212 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 3277212 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 3277213 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 3277214 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 3277501 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestReplicaProperties_D08670B1FAA54301-001/control-001/cores/collection1
   [junit4]   2> 3277502 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 3277504 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1157e1a5{/,null,AVAILABLE}
   [junit4]   2> 3277504 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.e.j.s.ServerConnector Started ServerConnector@2f10d173{HTTP/1.1}{127.0.0.1:57326}
   [junit4]   2> 3277504 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.e.j.s.Server Started @3280459ms
   [junit4]   2> 3277504 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestReplicaProperties_D08670B1FAA54301-001/tempDir-001/control/data, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestReplicaProperties_D08670B1FAA54301-001/control-001/cores, hostPort=57326}
   [junit4]   2> 3277504 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@4e0add57
   [junit4]   2> 3277505 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestReplicaProperties_D08670B1FAA54301-001/control-001/'
   [junit4]   2> 3277505 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 3277505 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 3277521 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3277521 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3277523 INFO  (zkCallback-2473-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@450f73e5 name:ZooKeeperConnection Watcher:127.0.0.1:47499/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3277523 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3277523 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3277524 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 3277524 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestReplicaProperties_D08670B1FAA54301-001/control-001/solr.xml
   [junit4]   2> 3277538 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestReplicaProperties_D08670B1FAA54301-001/control-001/cores
   [junit4]   2> 3277538 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.CoreContainer New CoreContainer 1578509904
   [junit4]   2> 3277538 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestReplicaProperties_D08670B1FAA54301-001/control-001/]
   [junit4]   2> 3277538 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestReplicaProperties_D08670B1FAA54301-001/control-001/lib
   [junit4]   2> 3277538 WARN  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestReplicaProperties_D08670B1FAA54301-001/control-001/lib).
   [junit4]   2> 3277546 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 3277547 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 3277548 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 3277548 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 3277548 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:47499/solr
   [junit4]   2> 3277548 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 3277548 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3277549 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3277550 INFO  (zkCallback-2475-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7f65dd9f name:ZooKeeperConnection Watcher:127.0.0.1:47499 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3277550 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3277551 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3277552 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [n:127.0.0.1:57326_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3277553 INFO  (zkCallback-2476-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@17ea5ef5 name:ZooKeeperConnection Watcher:127.0.0.1:47499/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3277553 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [n:127.0.0.1:57326_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3277554 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [n:127.0.0.1:57326_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 3277556 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [n:127.0.0.1:57326_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 3277559 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [n:127.0.0.1:57326_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 3277560 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [n:127.0.0.1:57326_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 3277562 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [n:127.0.0.1:57326_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 3277565 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [n:127.0.0.1:57326_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 3277566 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [n:127.0.0.1:57326_    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 3277568 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [n:127.0.0.1:57326_    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 3277569 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [n:127.0.0.1:57326_    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 3277570 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [n:127.0.0.1:57326_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:57326_
   [junit4]   2> 3277570 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [n:127.0.0.1:57326_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:57326_
   [junit4]   2> 3277572 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [n:127.0.0.1:57326_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 3277573 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [n:127.0.0.1:57326_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 3277574 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [n:127.0.0.1:57326_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 3277575 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [n:127.0.0.1:57326_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94771138685435908-127.0.0.1:57326_-n_0000000000
   [junit4]   2> 3277575 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [n:127.0.0.1:57326_    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:57326_
   [junit4]   2> 3277575 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [n:127.0.0.1:57326_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 3277576 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [n:127.0.0.1:57326_    ] o.a.s.c.Overseer Overseer (id=94771138685435908-127.0.0.1:57326_-n_0000000000) starting
   [junit4]   2> 3277578 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [n:127.0.0.1:57326_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 3277584 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [n:127.0.0.1:57326_    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 3277584 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [n:127.0.0.1:57326_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 3277584 INFO  (OverseerCollectionConfigSetProcessor-94771138685435908-127.0.0.1:57326_-n_0000000000) [n:127.0.0.1:57326_    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 3277584 INFO  (OverseerStateUpdate-94771138685435908-127.0.0.1:57326_-n_0000000000) [n:127.0.0.1:57326_    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 3277609 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [n:127.0.0.1:57326_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 3277609 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [n:127.0.0.1:57326_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 3277610 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [n:127.0.0.1:57326_    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestReplicaProperties_D08670B1FAA54301-001/control-001/cores
   [junit4]   2> 3277611 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [n:127.0.0.1:57326_    ] o.a.s.c.SolrCore Created CoreDescriptor: {configSetProperties=configsetprops.json, loadOnStartup=true, collection=control_collection, name=collection1, shard=, config=solrconfig.xml, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestReplicaProperties_D08670B1FAA54301-001/control-001/cores/collection1/, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestReplicaProperties_D08670B1FAA54301-001/control-001/cores/collection1, coreNodeName=, dataDir=data/, transient=false, schema=schema.xml}
   [junit4]   2> 3277611 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [n:127.0.0.1:57326_    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestReplicaProperties_D08670B1FAA54301-001/control-001/cores/collection1/
   [junit4]   2> 3277611 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [n:127.0.0.1:57326_    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 3277612 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [n:127.0.0.1:57326_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 3277612 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [n:127.0.0.1:57326_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 3277612 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 3277612 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 3277614 INFO  (OverseerStateUpdate-94771138685435908-127.0.0.1:57326_-n_0000000000) [n:127.0.0.1:57326_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:57326",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "node_name":"127.0.0.1:57326_",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 3277614 INFO  (OverseerStateUpdate-94771138685435908-127.0.0.1:57326_-n_0000000000) [n:127.0.0.1:57326_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:57326",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "node_name":"127.0.0.1:57326_",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 3277615 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 3277615 INFO  (OverseerStateUpdate-94771138685435908-127.0.0.1:57326_-n_0000000000) [n:127.0.0.1:57326_    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 3277615 INFO  (OverseerStateUpdate-94771138685435908-127.0.0.1:57326_-n_0000000000) [n:127.0.0.1:57326_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 3277617 INFO  (zkCallback-2476-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3278615 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 3278615 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 3278616 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 3278616 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 3278617 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 3278617 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestReplicaProperties_D08670B1FAA54301-001/control-001/cores/collection1/'
   [junit4]   2> 3278617 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 3278617 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 3278632 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 3278638 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 3278651 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 3278680 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 3278681 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 3278687 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 3278904 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 3278906 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 3278907 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 3278910 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 3278929 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 3278929 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 3278929 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 3278929 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestReplicaProperties_D08670B1FAA54301-001/control-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 3278929 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@5e1a3715
   [junit4]   2> 3278930 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestReplicaProperties_D08670B1FAA54301-001/control-001/cores/collection1/data
   [junit4]   2> 3278930 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestReplicaProperties_D08670B1FAA54301-001/control-001/cores/collection1/data/index/
   [junit4]   2> 3278930 WARN  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestReplicaProperties_D08670B1FAA54301-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 3278930 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestReplicaProperties_D08670B1FAA54301-001/control-001/cores/collection1/data/index
   [junit4]   2> 3278931 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=48, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.3183713194007609]
   [junit4]   2> 3278934 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(NIOFSDirectory@/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestReplicaProperties_D08670B1FAA54301-001/index-NIOFSDirectory-001 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6781a840),segFN=segments_1,generation=1}
   [junit4]   2> 3278934 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 3278934 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestReplicaProperties_D08670B1FAA54301-001/control-001/cores/collection1/data/
   [junit4]   2> 3278935 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestReplicaProperties_D08670B1FAA54301-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 3278938 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 3278938 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 3278939 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 3278939 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 3278939 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 3278939 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 3278939 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 3278939 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 3278939 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 3278942 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 3278943 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 3278945 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 3278946 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 3278949 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: standard,/admin/plugins,/update,/admin/mbeans,/admin/segments,/schema,/admin/ping,/admin/logging,/admin/file,/update/json,/get,/update/json/docs,/update/csv,/admin/properties,/replication,/config,/admin/system,/admin/threads,/admin/luke
   [junit4]   2> 3278951 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 3278951 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 3278951 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 3278952 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 3278952 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 3278952 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=18, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
   [junit4]   2> 3278953 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(NIOFSDirectory@/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestReplicaProperties_D08670B1FAA54301-001/index-NIOFSDirectory-001 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6781a840),segFN=segments_1,generation=1}
   [junit4]   2> 3278953 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 3278954 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@2ba9a561[collection1] main
   [junit4]   2> 3278954 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 3278955 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 3278955 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 3278955 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 3278955 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 3278955 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 3278956 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 3278956 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 3278956 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 3278956 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 3278956 INFO  (searcherExecutor-6371-thread-1-processing-c:control_collection r:core_node1 s:shard1 x:collection1 n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@2ba9a561[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 3278957 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 3278957 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 3278957 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 3278957 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1516338220933054464
   [junit4]   2> 3278959 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 2.0ms to seed version buckets with highest version 1516338220933054464
   [junit4]   2> 3278959 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 3278959 INFO  (coreLoadExecutor-6370-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 3278960 INFO  (coreZkRegister-6364-thread-1-processing-c:control_collection r:core_node1 s:shard1 x:collection1 n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:57326 collection:control_collection shard:shard1
   [junit4]   2> 3278961 INFO  (coreZkRegister-6364-thread-1-processing-c:control_collection r:core_node1 s:shard1 x:collection1 n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 3278964 INFO  (coreZkRegister-6364-thread-1-processing-c:control_collection r:core_node1 s:shard1 x:collection1 n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94771138685435908-core_node1-n_0000000000
   [junit4]   2> 3278965 INFO  (coreZkRegister-6364-thread-1-processing-c:control_collection r:core_node1 s:shard1 x:collection1 n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 3278966 INFO  (coreZkRegister-6364-thread-1-processing-c:control_collection r:core_node1 s:shard1 x:collection1 n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 3278967 INFO  (coreZkRegister-6364-thread-1-processing-c:control_collection r:core_node1 s:shard1 x:collection1 n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 3278967 INFO  (coreZkRegister-6364-thread-1-processing-c:control_collection r:core_node1 s:shard1 x:collection1 n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:57326/collection1/
   [junit4]   2> 3278967 INFO  (coreZkRegister-6364-thread-1-processing-c:control_collection r:core_node1 s:shard1 x:collection1 n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 3278967 INFO  (coreZkRegister-6364-thread-1-processing-c:control_collection r:core_node1 s:shard1 x:collection1 n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:57326/collection1/ has no replicas
   [junit4]   2> 3278967 INFO  (coreZkRegister-6364-thread-1-processing-c:control_collection r:core_node1 s:shard1 x:collection1 n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:57326/collection1/ shard1
   [junit4]   2> 3278967 INFO  (OverseerStateUpdate-94771138685435908-127.0.0.1:57326_-n_0000000000) [n:127.0.0.1:57326_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 3278967 INFO  (coreZkRegister-6364-thread-1-processing-c:control_collection r:core_node1 s:shard1 x:collection1 n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 3278970 INFO  (coreZkRegister-6364-thread-1-processing-c:control_collection r:core_node1 s:shard1 x:collection1 n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 3278972 INFO  (OverseerStateUpdate-94771138685435908-127.0.0.1:57326_-n_0000000000) [n:127.0.0.1:57326_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:57326",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 1
   [junit4]   2> 3279075 INFO  (zkCallback-2476-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3279114 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3279114 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3279116 INFO  (zkCallback-2478-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@73be9c name:ZooKeeperConnection Watcher:127.0.0.1:47499/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3279116 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3279116 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3279116 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 3279119 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 3279119 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with stateFormat=2
   [junit4]   2> 3279119 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3279122 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3279123 INFO  (zkCallback-2479-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@59848a94 name:ZooKeeperConnection Watcher:127.0.0.1:47499/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3279123 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3279123 INFO  (coreZkRegister-6364-thread-1-processing-c:control_collection r:core_node1 s:shard1 x:collection1 n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:57326/collection1/ and leader is http://127.0.0.1:57326/collection1/
   [junit4]   2> 3279123 INFO  (coreZkRegister-6364-thread-1-processing-c:control_collection r:core_node1 s:shard1 x:collection1 n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:57326
   [junit4]   2> 3279123 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3279123 INFO  (coreZkRegister-6364-thread-1-processing-c:control_collection r:core_node1 s:shard1 x:collection1 n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 3279123 INFO  (coreZkRegister-6364-thread-1-processing-c:control_collection r:core_node1 s:shard1 x:collection1 n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 3279123 INFO  (coreZkRegister-6364-thread-1-processing-c:control_collection r:core_node1 s:shard1 x:collection1 n:127.0.0.1:57326_) [n:127.0.0.1:57326_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 3279126 INFO  (OverseerStateUpdate-94771138685435908-127.0.0.1:57326_-n_0000000000) [n:127.0.0.1:57326_    ] o.a.s.c.Overseer processMessage: queueSize: 2, message = {
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:57326",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "node_name":"127.0.0.1:57326_",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "operation":"state"} current state version: 2
   [junit4]   2> 3279127 INFO  (OverseerStateUpdate-94771138685435908-127.0.0.1:57326_-n_0000000000) [n:127.0.0.1:57326_    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:57326",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "node_name":"127.0.0.1:57326_",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 3279128 INFO  (OverseerStateUpdate-94771138685435908-127.0.0.1:57326_-n_0000000000) [n:127.0.0.1:57326_    ] o.a.s.c.Overseer processMessage: queueSize: 2, message = {
   [junit4]   2>   "operation":"create",
   [junit4]   2>   "name":"collection1",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "stateFormat":"2"} current state version: 2
   [junit4]   2> 3279129 INFO  (OverseerStateUpdate-94771138685435908-127.0.0.1:57326_-n_0000000000) [n:127.0.0.1:57326_    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 3279131 INFO  (zkCallback-2476-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3279131 INFO  (zkCallback-2478-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3279232 INFO  (OverseerStateUpdate-94771138685435908-127.0.0.1:57326_-n_0000000000) [n:127.0.0.1:57326_    ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/collection1/state.json
   [junit4]   2> 3279426 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestReplicaProperties_D08670B1FAA54301-001/shard-1-001/cores/collection1
   [junit4]   2> 3279427 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestReplicaProperties_D08670B1FAA54301-001/shard-1-001
   [junit4]   2> 3279428 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 3279431 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5d431817{/,null,AVAILABLE}
   [junit4]   2> 3279432 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.e.j.s.ServerConnector Started ServerConnector@5fe2bdd6{HTTP/1.1}{127.0.0.1:51380}
   [junit4]   2> 3279432 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.e.j.s.Server Started @3282387ms
   [junit4]   2> 3279432 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solrconfig=solrconfig.xml, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestReplicaProperties_D08670B1FAA54301-001/tempDir-001/jetty1, hostPort=51380, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestReplicaProperties_D08670B1FAA54301-001/shard-1-001/cores, hostContext=/}
   [junit4]   2> 3279432 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@4e0add57
   [junit4]   2> 3279432 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestReplicaProperties_D08670B1FAA54301-001/shard-1-001/'
   [junit4]   2> 3279433 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 3279433 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 3279448 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3279449 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3279450 INFO  (zkCallback-2480-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@858e8fb name:ZooKeeperConnection Watcher:127.0.0.1:47499/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3279451 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3279451 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3279452 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 3279452 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestReplicaProperties_D08670B1FAA54301-001/shard-1-001/solr.xml
   [junit4]   2> 3279465 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestReplicaProperties_D08670B1FAA54301-001/shard-1-001/cores
   [junit4]   2> 3279465 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.CoreContainer New CoreContainer 805970313
   [junit4]   2> 3279465 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestReplicaProperties_D08670B1FAA54301-001/shard-1-001/]
   [junit4]   2> 3279465 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestReplicaProperties_D08670B1FAA54301-001/shard-1-001/lib
   [junit4]   2> 3279465 WARN  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestReplicaProperties_D08670B1FAA54301-001/shard-1-001/lib).
   [junit4]   2> 3279473 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 3279474 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 3279475 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 3279475 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 3279475 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:47499/solr
   [junit4]   2> 3279475 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 3279475 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3279476 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3279477 INFO  (zkCallback-2482-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6923f55f name:ZooKeeperConnection Watcher:127.0.0.1:47499 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3279477 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3279477 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3279479 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [n:127.0.0.1:51380_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3279480 INFO  (zkCallback-2483-thread-1-processing-n:127.0.0.1:51380_) [n:127.0.0.1:51380_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@57cc1883 name:ZooKeeperConnection Watcher:127.0.0.1:47499/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3279480 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [n:127.0.0.1:51380_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3279485 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [n:127.0.0.1:51380_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 3280490 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [n:127.0.0.1:51380_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:51380_
   [junit4]   2> 3280491 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [n:127.0.0.1:51380_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:51380_
   [junit4]   2> 3280492 INFO  (zkCallback-2476-thread-1-processing-n:127.0.0.1:57326_) [n:127.0.0.1:57326_    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3280492 INFO  (zkCallback-2478-thread-1) [    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3280492 INFO  (zkCallback-2483-thread-1-processing-n:127.0.0.1:51380_) [n:127.0.0.1:51380_    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3280494 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [n:127.0.0.1:51380_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 3280495 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [n:127.0.0.1:51380_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94771138685435913-127.0.0.1:51380_-n_0000000001
   [junit4]   2> 3280496 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [n:127.0.0.1:51380_    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/94771138685435908-127.0.0.1:57326_-n_0000000000 to know if I could be the leader
   [junit4]   2> 3280505 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [n:127.0.0.1:51380_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 3280505 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [n:127.0.0.1:51380_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 3280506 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [n:127.0.0.1:51380_    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestReplicaProperties_D08670B1FAA54301-001/shard-1-001/cores
   [junit4]   2> 3280507 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [n:127.0.0.1:51380_    ] o.a.s.c.SolrCore Created CoreDescriptor: {dataDir=data/, coreNodeName=, shard=, transient=false, config=solrconfig.xml, schema=schema.xml, collection=collection1, name=collection1, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestReplicaProperties_D08670B1FAA54301-001/shard-1-001/cores/collection1/, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestReplicaProperties_D08670B1FAA54301-001/shard-1-001/cores/collection1, loadOnStartup=true, configSetProperties=configsetprops.json}
   [junit4]   2> 3280507 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [n:127.0.0.1:51380_    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestReplicaProperties_D08670B1FAA54301-001/shard-1-001/cores/collection1/
   [junit4]   2> 3280507 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [n:127.0.0.1:51380_    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 3280508 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [n:127.0.0.1:51380_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 3280508 INFO  (TEST-TestReplicaProperties.test-seed#[D08670B1FAA54301]) [n:127.0.0.1:51380_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 3280508 INFO  (coreLoadExecutor-6381-thread-1-processing-n:127.0.0.1:51380_) [n:127.0.0.1:51380_ c:collection1   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 3280508 INFO  (coreLoadExecutor-6381-thread-1-processing-n:127.0.0.1:51380_) [n:127.0.0.1:51380_ c:collection1   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 3280509 INFO  (coreLoadExecutor-6381-thread-1-processing-n:127.0.0.1:51380_) [n:127.0.0.1:51380_ c:collection1   x:collection1] o.a.s.c.ZkController Registering watch for collection collection1
   [junit4]   2> 3280510 INFO  (coreLoadExecutor-6381-thread-1-processing-n:127.0.0.1:51380_) [n:127.0.0.1:51380_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader addZkWatch collection1
   [junit4]   2> 3280510 INFO  (OverseerStateUpdate-94771138685435908-127.0.0.1:57326_-n_0000000000) [n:127.0.0.1:57326_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:51380",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "node_name":"127.0.0.1:51380_",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "operation":"state"} current state version: 3
   [junit4]   2> 3280510 INFO  (coreLoadExecutor-6381-thread-1-processing-n:127.0.0.1:51380_) [n:127.0.0.1:51380_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Add data for collection1 ver 0 
   [junit4]   2> 3280510 INFO  (coreLoadExecutor-6381-thread-1-processing-n:127.0.0.1:51380_) [n:127.0.0.1:51380_ c:collection1   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 3280510 INFO  (OverseerStateUpdate-94771138685435908-127.0.0.1:57326_-n_0000000000) [n:127.0.0.1:57326_    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:51380",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "node_name":"127.0.0.1:51380_",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 3280510 INFO  (OverseerStateUpdate-94771138685435908-127.0.0.1:57326_-n_0000000000) [n:127.0.0.1:57326_    ] o.a.s.c.o.ReplicaMutator Collection already exists with numShards=2
   [junit4]   2> 3280510 INFO  (OverseerStateUpdate-94771138685435908-127.0.0.1:57326_-n_0000000000) [n:127.0.0.1:57326_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard2
   [junit4]   2> 3280613 INFO  (OverseerStateUpdate-94771138685435908-127.0.0.1:57326_-n_0000000000) [n:127.0.0.1:57326_    ] o.a.s.c.o.ZkStateWriter going to update_collection /collections/collection1/state.json version: 0
   [junit4]   2> 3280613 INFO  (zkCallback-2483-thread-1-processing-n:127.0.0.1:51380_) [n:127.0.0.1:51380_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json for collection collection1 has occurred - updating... (live nodes size: 2)
   [junit4]   2> 3280614 INFO  (zkCallback-2483-thread-1-processing-n:127.0.0.1:51380_) [n:127.0.0.1:51380_    ] o.a.s.c.c.ZkStateReader Updating data for collection1 from 0 to 1 
   [junit4]   2> 3281510 INFO  (coreLoadExecutor-6381-thread-1-processing-n:127.0.0.1:51380_) [n:127.0.0.1:51380_ c:collection1   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 3281510 INFO  (coreLoadExecutor-6381-thread-1-processing-n:127.0.0.1:51380_) [n:127.0.0.1:51380_ c:collection1   x:collection1] o.a.s.c.ZkController Check for collection zkNode:collection1
   [junit4]   2> 3281511 INFO  (coreLoadExecutor-6381-thread-1-processing-n:127.0.0.1:51380_) [n:127.0.0.1:51380_ c:collection1   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 3281511 INFO  (coreLoadExecutor-6381-thread-1-processing-n:127.0.0.1:51380_) [n:127.0.0.1:51380_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/collection1
   [junit4]   2> 3281512 INFO  (coreLoadExecutor-6381-thread-1-processing-n:127.0.0.1:51380_) [n:127.0.0.1:51380_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 3281512 INFO  (coreLoadExecutor-6381-thread-1-processing-n:127.0.0.1:51380_) [n:127.0.0.1:51380_ c:collection1   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestReplicaProperties_D08670B1FAA54301-001/shard-1-001/cores/collection1/'
   [junit4]   2> 3281512 INFO  (coreLoadExecutor-6381-thread-1-processing-n:127.0.0.1:51380_) [n:127.0.0.1:51380_ c:collection1   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 3281512 INFO  (coreLoadExecutor-6381-thread-1-processing-n:127.0.0.1:51380_) [n:127.0.0.1:51380_ c:collection1   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 3281527 INFO  (coreLoadExecutor-6381-thread-1-processing-n:127.0.0.1:51380_) [n:127.0.0.

[...truncated too long message...]

 "core":"testcollection_shard3_replica3",
   [junit4]    >           "state":"active"}}},
   [junit4]    >     "shard4":{
   [junit4]    >       "range":"19990000-4ccbffff",
   [junit4]    >       "state":"active",
   [junit4]    >       "replicas":{
   [junit4]    >         "core_node4":{
   [junit4]    >           "base_url":"http://127.0.0.1:51380",
   [junit4]    >           "node_name":"127.0.0.1:51380_",
   [junit4]    >           "core":"testcollection_shard4_replica3",
   [junit4]    >           "state":"active",
   [junit4]    >           "leader":"true"},
   [junit4]    >         "core_node11":{
   [junit4]    >           "base_url":"http://127.0.0.1:45841",
   [junit4]    >           "node_name":"127.0.0.1:45841_",
   [junit4]    >           "core":"testcollection_shard4_replica2",
   [junit4]    >           "state":"active",
   [junit4]    >           "property.preferredleader":"true"},
   [junit4]    >         "core_node12":{
   [junit4]    >           "base_url":"http://127.0.0.1:34598",
   [junit4]    >           "node_name":"127.0.0.1:34598_",
   [junit4]    >           "core":"testcollection_shard4_replica1",
   [junit4]    >           "state":"active"}}},
   [junit4]    >     "shard5":{
   [junit4]    >       "range":"4ccc0000-7fffffff",
   [junit4]    >       "state":"active",
   [junit4]    >       "replicas":{
   [junit4]    >         "core_node3":{
   [junit4]    >           "base_url":"http://127.0.0.1:42012",
   [junit4]    >           "node_name":"127.0.0.1:42012_",
   [junit4]    >           "core":"testcollection_shard5_replica2",
   [junit4]    >           "state":"active",
   [junit4]    >           "leader":"true"},
   [junit4]    >         "core_node6":{
   [junit4]    >           "base_url":"http://127.0.0.1:57326",
   [junit4]    >           "node_name":"127.0.0.1:57326_",
   [junit4]    >           "core":"testcollection_shard5_replica1",
   [junit4]    >           "state":"active",
   [junit4]    >           "property.preferredleader":"true"},
   [junit4]    >         "core_node13":{
   [junit4]    >           "base_url":"http://127.0.0.1:34598",
   [junit4]    >           "node_name":"127.0.0.1:34598_",
   [junit4]    >           "core":"testcollection_shard5_replica3",
   [junit4]    >           "state":"active"}}}},
   [junit4]    >   "replicationFactor":"3",
   [junit4]    >   "autoAddReplicas":"false",
   [junit4]    >   "maxShardsPerNode":"16",
   [junit4]    >   "router":{"name":"compositeId"}}
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([D08670B1FAA54301:58D24F6B54592EF9]:0)
   [junit4]    > 	at org.apache.solr.cloud.ReplicaPropertiesBase.verifyUnique(ReplicaPropertiesBase.java:176)
   [junit4]    > 	at org.apache.solr.cloud.ReplicaPropertiesBase.verifyUniqueAcrossCollection(ReplicaPropertiesBase.java:107)
   [junit4]    > 	at org.apache.solr.cloud.TestReplicaProperties.clusterAssignPropertyTest(TestReplicaProperties.java:108)
   [junit4]    > 	at org.apache.solr.cloud.TestReplicaProperties.test(TestReplicaProperties.java:70)
   [junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:963)
   [junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:938)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 3341181 INFO  (SUITE-TestReplicaProperties-seed#[D08670B1FAA54301]-worker) [n:127.0.0.1:42012_ c:testcollection s:shard3 r:core_node14 x:testcollection_shard3_replica3] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestReplicaProperties_D08670B1FAA54301-001
   [junit4]   2> Oct 29, 2015 12:27:58 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene54): {}, docValues:{}, sim=RandomSimilarityProvider(queryNorm=false,coord=crazy): {}, locale=ar_DZ, timezone=America/North_Dakota/Beulah
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_72 (64-bit)/cpus=4,threads=1,free=116519584,total=524288000
   [junit4]   2> NOTE: All tests run in this JVM: [TestFieldTypeCollectionResource, TestSystemIdResolver, TestExactSharedStatsCache, TestCryptoKeys, AliasIntegrationTest, ShowFileRequestHandlerTest, TestNRTOpen, QueryParsingTest, TestReRankQParserPlugin, TestChildDocTransformer, LeaderInitiatedRecoveryOnShardRestartTest, AutoCommitTest, HLLSerializationTest, TestDefaultSearchFieldResource, MergeStrategyTest, TestHashQParserPlugin, TestZkChroot, XmlUpdateRequestHandlerTest, TestSimpleQParserPlugin, DirectUpdateHandlerOptimizeTest, InfoHandlerTest, SuggesterTSTTest, MultiThreadedOCPTest, SharedFSAutoReplicaFailoverTest, CurrencyFieldXmlFileTest, TestCSVResponseWriter, TestSmileRequest, TestCustomDocTransformer, TestConfigSetsAPI, FieldAnalysisRequestHandlerTest, BinaryUpdateRequestHandlerTest, RulesTest, UUIDFieldTest, TestDynamicFieldCollectionResource, BadIndexSchemaTest, LukeRequestHandlerTest, TestSolrQueryParser, BadComponentTest, TestPostingsSolrHighlighter, DistributedSuggestComponentTest, DistributedQueryComponentCustomSortTest, BlockDirectoryTest, TestCharFilters, SuggesterWFSTTest, TestCloudInspectUtil, TestPHPSerializedResponseWriter, TestSolrIndexConfig, TestComplexPhraseQParserPlugin, HighlighterConfigTest, TestFastWriter, UpdateParamsTest, PluginInfoTest, HttpPartitionTest, HardAutoCommitTest, OverseerRolesTest, SuggesterTest, SampleTest, DocValuesMissingTest, DirectSolrConnectionTest, TestLRUCache, TestJsonRequest, TestCollationField, TestCloudManagedSchemaConcurrent, TestElisionMultitermQuery, QueryResultKeyTest, TestReversedWildcardFilterFactory, TestSolrDeletionPolicy1, TestSolrQueryParserResource, TestCloudPivotFacet, TestFileDictionaryLookup, StatelessScriptUpdateProcessorFactoryTest, DeleteLastCustomShardedReplicaTest, TestGroupingSearch, PrimitiveFieldTypeTest, CoreAdminHandlerTest, SolrCoreCheckLockOnStartupTest, TestStressReorder, TestScoreJoinQPScore, DocValuesMultiTest, TestConfigSetImmutable, TestFastLRUCache, RuleEngineTest, AnalyticsMergeStrategyTest, HdfsWriteToMultipleCollectionsTest, TestSort, SharedFSAutoReplicaFailoverUtilsTest, OpenCloseCoreStressTest, LeaderElectionTest, ZkSolrClientTest, TestRecovery, TestHashPartitioner, DistributedSpellCheckComponentTest, TermVectorComponentDistributedTest, TestRealTimeGet, TestJoin, TestReload, TestRangeQuery, SimpleFacetsTest, TestSolr4Spatial, StatsComponentTest, SpellCheckComponentTest, PeerSyncTest, TestFiltering, BasicFunctionalityTest, TestLazyCores, DirectUpdateHandlerTest, TestIndexSearcher, HighlighterTest, SoftAutoCommitTest, SolrIndexSplitterTest, SimplePostToolTest, TestCoreDiscovery, TestExtendedDismaxParser, SpellCheckCollatorTest, SolrRequestParserTest, SpatialFilterTest, PolyFieldTest, NoCacheHeaderTest, TestPseudoReturnFields, TestUpdate, FieldMutatingUpdateProcessorTest, TestAtomicUpdateErrorCases, DefaultValueUpdateProcessorTest, SortByFunctionTest, TestRemoteStreaming, SolrInfoMBeanTest, DebugComponentTest, CacheHeaderTest, IndexBasedSpellCheckerTest, TestSurroundQueryParser, DisMaxRequestHandlerTest, TestWriterPerf, DocumentAnalysisRequestHandlerTest, TestQueryTypes, TermsComponentTest, RequestHandlersTest, TestValueSourceCache, TermVectorComponentTest, TestIndexingPerformance, TestArbitraryIndexDir, IndexSchemaRuntimeFieldTest, RegexBoostProcessorTest, SolrPluginUtilsTest, TestJmxIntegration, UpdateRequestProcessorFactoryTest, TestPartialUpdateDeduplication, TestComponentsName, SearchHandlerTest, TestLFUCache, TestQuerySenderListener, AlternateDirectoryTest, TestQuerySenderNoQuery, ResponseLogComponentTest, TestStressRecovery, NotRequiredUniqueKeyTest, TestPhraseSuggestions, TestXIncludeConfig, TestSweetSpotSimilarityFactory, TestLMDirichletSimilarityFactory, TestBM25SimilarityFactory, TestJmxMonitoredMap, ResourceLoaderTest, URLClassifyProcessorTest, ChaosMonkeyNothingIsSafeTest, TestCrossCoreJoin, TestCursorMarkWithoutUniqueKey, TestDistributedMissingSort, TestEmbeddedSolrServerConstructors, ActionThrottleTest, CollectionReloadTest, ConcurrentDeleteAndCreateCollectionTest, DeleteReplicaTest, DistribCursorPagingTest, MigrateRouteKeyTest, SSLMigrationTest, SimpleCollectionCreateDeleteTest, TestLeaderElectionZkExpiry, TestMiniSolrCloudCluster, TestReplicaProperties]
   [junit4] Completed [435/543] on J2 in 64.65s, 1 test, 1 failure <<< FAILURES!

[...truncated 363 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 543 suites (7 ignored), 2147 tests, 1 failure, 31 ignored (18 assumptions) [seed: D08670B1FAA54301]

Total time: 204 minutes 32 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 999 - Still Failing

[Apache Jenkins Server <je...@builds.apache.org>]

Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/999/

2 tests failed.
FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Error from server at http://127.0.0.1:58084/wb_dlc/o: Could not load collection from ZK:halfcollectionblocker

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:58084/wb_dlc/o: Could not load collection from ZK:halfcollectionblocker
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.makeRequest(CollectionsAPIDistributedZkTest.java:302)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testErrorHandling(CollectionsAPIDistributedZkTest.java:419)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test(CollectionsAPIDistributedZkTest.java:169)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1665)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:864)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:900)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:914)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:963)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:938)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:873)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:775)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:809)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:820)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR

Error Message:
Captured an uncaught exception in thread: Thread[id=168934, name=coreZkRegister-6548-thread-1, state=RUNNABLE, group=TGRP-LeaderInitiatedRecoveryOnShardRestartTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=168934, name=coreZkRegister-6548-thread-1, state=RUNNABLE, group=TGRP-LeaderInitiatedRecoveryOnShardRestartTest]
Caused by: java.lang.AssertionError
	at __randomizedtesting.SeedInfo.seed([DC6E135E9ED086E8]:0)
	at org.apache.solr.cloud.ZkController.updateLeaderInitiatedRecoveryState(ZkController.java:2133)
	at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:434)
	at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)
	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)
	at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:346)
	at org.apache.solr.cloud.ZkController.joinElection(ZkController.java:1113)
	at org.apache.solr.cloud.ZkController.register(ZkController.java:926)
	at org.apache.solr.cloud.ZkController.register(ZkController.java:881)
	at org.apache.solr.core.ZkContainer$2.run(ZkContainer.java:183)
	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 11261 lines...]
   [junit4] Suite: org.apache.solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_DC6E135E9ED086E8-001/init-core-data-001
   [junit4]   2> 3204461 INFO  (SUITE-LeaderInitiatedRecoveryOnShardRestartTest-seed#[DC6E135E9ED086E8]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /n/
   [junit4]   2> 3204465 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 3204465 INFO  (Thread-159406) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 3204465 INFO  (Thread-159406) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 3204565 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.ZkTestServer start zk server on port:49762
   [junit4]   2> 3204565 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3204566 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3204569 INFO  (zkCallback-2465-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@11591a5e name:ZooKeeperConnection Watcher:127.0.0.1:49762 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3204569 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3204569 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3204570 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 3204573 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3204574 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3204575 INFO  (zkCallback-2466-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@34d6f611 name:ZooKeeperConnection Watcher:127.0.0.1:49762/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3204575 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3204575 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3204575 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 3204577 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 3204579 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 3204580 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 3204581 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 3204582 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 3204584 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 3204584 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 3204586 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 3204587 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 3204588 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 3204589 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 3204590 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 3204590 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 3204592 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 3204592 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 3204594 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 3204594 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 3204596 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 3204596 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 3204598 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 3204598 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 3204600 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 3204600 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 3204602 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 3204602 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 3204915 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_DC6E135E9ED086E8-001/control-001/cores/collection1
   [junit4]   2> 3204917 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 3204919 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5a2e0305{/n,null,AVAILABLE}
   [junit4]   2> 3204919 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.e.j.s.ServerConnector Started ServerConnector@2c86b84{HTTP/1.1}{127.0.0.1:33937}
   [junit4]   2> 3204919 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.e.j.s.Server Started @3207923ms
   [junit4]   2> 3204920 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_DC6E135E9ED086E8-001/control-001/cores, hostContext=/n, hostPort=33937, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_DC6E135E9ED086E8-001/tempDir-001/control/data}
   [junit4]   2> 3204920 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@6b482747
   [junit4]   2> 3204920 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_DC6E135E9ED086E8-001/control-001/'
   [junit4]   2> 3204920 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 3204920 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 3204935 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3204936 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3204937 INFO  (zkCallback-2467-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@53e19508 name:ZooKeeperConnection Watcher:127.0.0.1:49762/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3204937 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3204938 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3204938 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 3204939 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_DC6E135E9ED086E8-001/control-001/solr.xml
   [junit4]   2> 3204952 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_DC6E135E9ED086E8-001/control-001/cores
   [junit4]   2> 3204952 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.CoreContainer New CoreContainer 608573074
   [junit4]   2> 3204952 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_DC6E135E9ED086E8-001/control-001/]
   [junit4]   2> 3204952 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_DC6E135E9ED086E8-001/control-001/lib
   [junit4]   2> 3204952 WARN  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_DC6E135E9ED086E8-001/control-001/lib).
   [junit4]   2> 3204960 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 3204961 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 3204961 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 3204961 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 3204962 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:49762/solr
   [junit4]   2> 3204962 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 3204962 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3204962 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3204964 INFO  (zkCallback-2469-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6b2187f8 name:ZooKeeperConnection Watcher:127.0.0.1:49762 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3204964 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3204964 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3204965 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [n:127.0.0.1:33937_n    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3204966 INFO  (zkCallback-2470-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6003761c name:ZooKeeperConnection Watcher:127.0.0.1:49762/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3204966 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [n:127.0.0.1:33937_n    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3204967 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [n:127.0.0.1:33937_n    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 3204969 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [n:127.0.0.1:33937_n    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 3204972 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [n:127.0.0.1:33937_n    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 3204974 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [n:127.0.0.1:33937_n    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 3204976 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [n:127.0.0.1:33937_n    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 3204977 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [n:127.0.0.1:33937_n    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 3204979 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [n:127.0.0.1:33937_n    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 3204980 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [n:127.0.0.1:33937_n    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 3204981 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [n:127.0.0.1:33937_n    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 3204982 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [n:127.0.0.1:33937_n    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:33937_n
   [junit4]   2> 3204983 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [n:127.0.0.1:33937_n    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:33937_n
   [junit4]   2> 3204984 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [n:127.0.0.1:33937_n    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 3204985 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [n:127.0.0.1:33937_n    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 3204986 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [n:127.0.0.1:33937_n    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 3204987 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [n:127.0.0.1:33937_n    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94764414492934148-127.0.0.1:33937_n-n_0000000000
   [junit4]   2> 3204987 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [n:127.0.0.1:33937_n    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:33937_n
   [junit4]   2> 3204987 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [n:127.0.0.1:33937_n    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 3204988 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [n:127.0.0.1:33937_n    ] o.a.s.c.Overseer Overseer (id=94764414492934148-127.0.0.1:33937_n-n_0000000000) starting
   [junit4]   2> 3204990 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [n:127.0.0.1:33937_n    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 3204996 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [n:127.0.0.1:33937_n    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 3204997 INFO  (OverseerCollectionConfigSetProcessor-94764414492934148-127.0.0.1:33937_n-n_0000000000) [n:127.0.0.1:33937_n    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 3204997 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [n:127.0.0.1:33937_n    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 3204997 INFO  (OverseerStateUpdate-94764414492934148-127.0.0.1:33937_n-n_0000000000) [n:127.0.0.1:33937_n    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 3205012 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [n:127.0.0.1:33937_n    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 3205012 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [n:127.0.0.1:33937_n    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 3205013 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [n:127.0.0.1:33937_n    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_DC6E135E9ED086E8-001/control-001/cores
   [junit4]   2> 3205014 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [n:127.0.0.1:33937_n    ] o.a.s.c.SolrCore Created CoreDescriptor: {absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_DC6E135E9ED086E8-001/control-001/cores/collection1/, coreNodeName=, collection=control_collection, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_DC6E135E9ED086E8-001/control-001/cores/collection1, name=collection1, config=solrconfig.xml, dataDir=data/, configSetProperties=configsetprops.json, loadOnStartup=true, schema=schema.xml, shard=, transient=false}
   [junit4]   2> 3205014 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [n:127.0.0.1:33937_n    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_DC6E135E9ED086E8-001/control-001/cores/collection1/
   [junit4]   2> 3205014 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [n:127.0.0.1:33937_n    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 3205015 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [n:127.0.0.1:33937_n    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0
   [junit4]   2> 3205015 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [n:127.0.0.1:33937_n    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 3205015 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 3205015 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 3205017 INFO  (OverseerStateUpdate-94764414492934148-127.0.0.1:33937_n-n_0000000000) [n:127.0.0.1:33937_n    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "shard":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:33937/n",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "state":"down",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "node_name":"127.0.0.1:33937_n"} current state version: 0
   [junit4]   2> 3205017 INFO  (OverseerStateUpdate-94764414492934148-127.0.0.1:33937_n-n_0000000000) [n:127.0.0.1:33937_n    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "shard":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:33937/n",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "state":"down",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "node_name":"127.0.0.1:33937_n"}
   [junit4]   2> 3205018 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 3205018 INFO  (OverseerStateUpdate-94764414492934148-127.0.0.1:33937_n-n_0000000000) [n:127.0.0.1:33937_n    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 3205018 INFO  (OverseerStateUpdate-94764414492934148-127.0.0.1:33937_n-n_0000000000) [n:127.0.0.1:33937_n    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 3205019 INFO  (zkCallback-2470-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3206018 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 3206018 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 3206019 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 3206019 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 3206019 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 3206019 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_DC6E135E9ED086E8-001/control-001/cores/collection1/'
   [junit4]   2> 3206020 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 3206020 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 3206039 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 3206048 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 3206062 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 3206128 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 3206129 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 3206137 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 3206398 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 3206406 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 3206408 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 3206417 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 3206420 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 3206423 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 3206424 WARN  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 3206424 WARN  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 3206424 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 3206425 WARN  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 3206425 WARN  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 3206426 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0
   [junit4]   2> 3206426 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 3206426 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 3206427 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_DC6E135E9ED086E8-001/control-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 3206427 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@31bfe7
   [junit4]   2> 3206427 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_DC6E135E9ED086E8-001/control-001/cores/collection1/data
   [junit4]   2> 3206427 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_DC6E135E9ED086E8-001/control-001/cores/collection1/data/index/
   [junit4]   2> 3206427 WARN  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_DC6E135E9ED086E8-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 3206428 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_DC6E135E9ED086E8-001/control-001/cores/collection1/data/index
   [junit4]   2> 3206428 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=46, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
   [junit4]   2> 3206428 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@4e6280cd lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@24f63aef),segFN=segments_1,generation=1}
   [junit4]   2> 3206429 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 3206429 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_DC6E135E9ED086E8-001/control-001/cores/collection1/data/
   [junit4]   2> 3206429 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_DC6E135E9ED086E8-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 3206432 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 3206432 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 3206433 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 3206433 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 3206433 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 3206433 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 3206434 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 3206434 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 3206434 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 3206437 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 3206438 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 3206440 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 3206441 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 3206444 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/system,/update,/admin/segments,/admin/threads,/config,/admin/properties,/update/csv,/update/json/docs,standard,/admin/luke,/admin/plugins,/replication,/update/json,/admin/ping,/admin/mbeans,/admin/logging,/get,/schema,/admin/file
   [junit4]   2> 3206446 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 3206446 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 3206446 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 3206447 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 3206447 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 3206448 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=12, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
   [junit4]   2> 3206448 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@4e6280cd lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@24f63aef),segFN=segments_1,generation=1}
   [junit4]   2> 3206448 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 3206448 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@2a3f771[collection1] main
   [junit4]   2> 3206449 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 3206449 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 3206449 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 3206450 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 3206450 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 3206450 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 3206450 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 3206451 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 3206451 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 3206451 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 3206451 INFO  (searcherExecutor-6500-thread-1-processing-s:shard1 x:collection1 c:control_collection r:core_node1 n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@2a3f771[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 3206451 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 3206452 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 3206452 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 3206452 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1516230633968369664
   [junit4]   2> 3206454 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 2.0ms to seed version buckets with highest version 1516230633968369664
   [junit4]   2> 3206454 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 3206455 INFO  (coreLoadExecutor-6499-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 3206455 INFO  (coreZkRegister-6493-thread-1-processing-s:shard1 x:collection1 c:control_collection r:core_node1 n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:33937/n collection:control_collection shard:shard1
   [junit4]   2> 3206456 INFO  (coreZkRegister-6493-thread-1-processing-s:shard1 x:collection1 c:control_collection r:core_node1 n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 3206459 INFO  (coreZkRegister-6493-thread-1-processing-s:shard1 x:collection1 c:control_collection r:core_node1 n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94764414492934148-core_node1-n_0000000000
   [junit4]   2> 3206460 INFO  (coreZkRegister-6493-thread-1-processing-s:shard1 x:collection1 c:control_collection r:core_node1 n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 3206461 INFO  (coreZkRegister-6493-thread-1-processing-s:shard1 x:collection1 c:control_collection r:core_node1 n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 3206462 INFO  (coreZkRegister-6493-thread-1-processing-s:shard1 x:collection1 c:control_collection r:core_node1 n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 3206462 INFO  (coreZkRegister-6493-thread-1-processing-s:shard1 x:collection1 c:control_collection r:core_node1 n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:33937/n/collection1/
   [junit4]   2> 3206462 INFO  (OverseerStateUpdate-94764414492934148-127.0.0.1:33937_n-n_0000000000) [n:127.0.0.1:33937_n    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 3206462 INFO  (coreZkRegister-6493-thread-1-processing-s:shard1 x:collection1 c:control_collection r:core_node1 n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 3206462 INFO  (coreZkRegister-6493-thread-1-processing-s:shard1 x:collection1 c:control_collection r:core_node1 n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:33937/n/collection1/ has no replicas
   [junit4]   2> 3206462 INFO  (coreZkRegister-6493-thread-1-processing-s:shard1 x:collection1 c:control_collection r:core_node1 n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:33937/n/collection1/ shard1
   [junit4]   2> 3206462 INFO  (coreZkRegister-6493-thread-1-processing-s:shard1 x:collection1 c:control_collection r:core_node1 n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 3206465 INFO  (coreZkRegister-6493-thread-1-processing-s:shard1 x:collection1 c:control_collection r:core_node1 n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 3206468 INFO  (OverseerStateUpdate-94764414492934148-127.0.0.1:33937_n-n_0000000000) [n:127.0.0.1:33937_n    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:33937/n",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 1
   [junit4]   2> 3206517 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3206518 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3206519 INFO  (zkCallback-2472-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@570fd5c3 name:ZooKeeperConnection Watcher:127.0.0.1:49762/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3206519 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3206519 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3206520 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 3206522 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 3206522 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with stateFormat=2
   [junit4]   2> 3206522 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3206523 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3206524 INFO  (zkCallback-2473-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@b2b1701 name:ZooKeeperConnection Watcher:127.0.0.1:49762/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3206524 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3206524 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3206526 INFO  (OverseerStateUpdate-94764414492934148-127.0.0.1:33937_n-n_0000000000) [n:127.0.0.1:33937_n    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"create",
   [junit4]   2>   "name":"collection1",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "stateFormat":"2"} current state version: 1
   [junit4]   2> 3206526 INFO  (OverseerStateUpdate-94764414492934148-127.0.0.1:33937_n-n_0000000000) [n:127.0.0.1:33937_n    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 3206527 INFO  (zkCallback-2470-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3206530 INFO  (zkCallback-2472-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3206568 INFO  (coreZkRegister-6493-thread-1-processing-s:shard1 x:collection1 c:control_collection r:core_node1 n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:33937/n/collection1/ and leader is http://127.0.0.1:33937/n/collection1/
   [junit4]   2> 3206568 INFO  (coreZkRegister-6493-thread-1-processing-s:shard1 x:collection1 c:control_collection r:core_node1 n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:33937/n
   [junit4]   2> 3206568 INFO  (coreZkRegister-6493-thread-1-processing-s:shard1 x:collection1 c:control_collection r:core_node1 n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 3206568 INFO  (coreZkRegister-6493-thread-1-processing-s:shard1 x:collection1 c:control_collection r:core_node1 n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 3206569 INFO  (coreZkRegister-6493-thread-1-processing-s:shard1 x:collection1 c:control_collection r:core_node1 n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 3206570 INFO  (OverseerStateUpdate-94764414492934148-127.0.0.1:33937_n-n_0000000000) [n:127.0.0.1:33937_n    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "base_url":"http://127.0.0.1:33937/n",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "state":"active",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "node_name":"127.0.0.1:33937_n"} current state version: 2
   [junit4]   2> 3206571 INFO  (OverseerStateUpdate-94764414492934148-127.0.0.1:33937_n-n_0000000000) [n:127.0.0.1:33937_n    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "base_url":"http://127.0.0.1:33937/n",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "state":"active",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "node_name":"127.0.0.1:33937_n"}
   [junit4]   2> 3206572 INFO  (OverseerStateUpdate-94764414492934148-127.0.0.1:33937_n-n_0000000000) [n:127.0.0.1:33937_n    ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/collection1/state.json
   [junit4]   2> 3206676 INFO  (zkCallback-2472-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3206676 INFO  (zkCallback-2470-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3206852 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_DC6E135E9ED086E8-001/shard-1-001/cores/collection1
   [junit4]   2> 3206853 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_DC6E135E9ED086E8-001/shard-1-001
   [junit4]   2> 3206854 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 3206856 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@37a6cfcd{/n,null,AVAILABLE}
   [junit4]   2> 3206856 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.e.j.s.ServerConnector Started ServerConnector@3bce107a{HTTP/1.1}{127.0.0.1:49376}
   [junit4]   2> 3206856 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.e.j.s.Server Started @3209860ms
   [junit4]   2> 3206856 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_DC6E135E9ED086E8-001/tempDir-001/jetty1, solrconfig=solrconfig.xml, hostContext=/n, hostPort=49376, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_DC6E135E9ED086E8-001/shard-1-001/cores}
   [junit4]   2> 3206857 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@6b482747
   [junit4]   2> 3206857 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_DC6E135E9ED086E8-001/shard-1-001/'
   [junit4]   2> 3206857 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 3206857 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 3206873 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3206874 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3206875 INFO  (zkCallback-2474-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@382dca3d name:ZooKeeperConnection Watcher:127.0.0.1:49762/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3206875 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3206875 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3206876 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 3206876 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_DC6E135E9ED086E8-001/shard-1-001/solr.xml
   [junit4]   2> 3206891 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_DC6E135E9ED086E8-001/shard-1-001/cores
   [junit4]   2> 3206892 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.CoreContainer New CoreContainer 794072415
   [junit4]   2> 3206892 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_DC6E135E9ED086E8-001/shard-1-001/]
   [junit4]   2> 3206892 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_DC6E135E9ED086E8-001/shard-1-001/lib
   [junit4]   2> 3206892 WARN  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_DC6E135E9ED086E8-001/shard-1-001/lib).
   [junit4]   2> 3206900 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 3206902 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 3206902 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 3206902 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 3206902 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:49762/solr
   [junit4]   2> 3206902 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 3206903 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 3206903 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3206904 INFO  (zkCallback-2476-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5f0f36d5 name:ZooKeeperConnection Watcher:127.0.0.1:49762 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3206905 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3206905 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 3206906 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [n:127.0.0.1:49376_n    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 3206907 INFO  (zkCallback-2477-thread-1-processing-n:127.0.0.1:49376_n) [n:127.0.0.1:49376_n    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7a8baeb8 name:ZooKeeperConnection Watcher:127.0.0.1:49762/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3206907 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [n:127.0.0.1:49376_n    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 3206913 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [n:127.0.0.1:49376_n    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 3207920 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [n:127.0.0.1:49376_n    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:49376_n
   [junit4]   2> 3207920 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [n:127.0.0.1:49376_n    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:49376_n
   [junit4]   2> 3207922 INFO  (zkCallback-2472-thread-1) [    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3207922 INFO  (zkCallback-2477-thread-1-processing-n:127.0.0.1:49376_n) [n:127.0.0.1:49376_n    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3207922 INFO  (zkCallback-2470-thread-1-processing-n:127.0.0.1:33937_n) [n:127.0.0.1:33937_n    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3207924 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [n:127.0.0.1:49376_n    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 3207924 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [n:127.0.0.1:49376_n    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94764414492934153-127.0.0.1:49376_n-n_0000000001
   [junit4]   2> 3207925 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [n:127.0.0.1:49376_n    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/94764414492934148-127.0.0.1:33937_n-n_0000000000 to know if I could be the leader
   [junit4]   2> 3207937 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [n:127.0.0.1:49376_n    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 3207937 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [n:127.0.0.1:49376_n    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 3207938 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [n:127.0.0.1:49376_n    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_DC6E135E9ED086E8-001/shard-1-001/cores
   [junit4]   2> 3207939 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [n:127.0.0.1:49376_n    ] o.a.s.c.SolrCore Created CoreDescriptor: {schema=schema.xml, coreNodeName=, configSetProperties=configsetprops.json, dataDir=data/, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_DC6E135E9ED086E8-001/shard-1-001/cores/collection1, name=collection1, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_DC6E135E9ED086E8-001/shard-1-001/cores/collection1/, shard=, transient=false, config=solrconfig.xml, collection=collection1, loadOnStartup=true}
   [junit4]   2> 3207939 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[DC6E135E9ED086E8]) [n:127.0.0.1:49376_n    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_DC6E135E9ED086E8-001/shard-1-001/c

[...truncated too long message...]

67/wb_dlc/o]
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([DC6E135E9ED086E8]:0)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:898)
   [junit4]    > Caused by: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:53022/wb_dlc/o, http://127.0.0.1:57344/wb_dlc/o, http://127.0.0.1:43240/wb_dlc/o, http://127.0.0.1:58084/wb_dlc/o, http://127.0.0.1:33567/wb_dlc/o]
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:352)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1574)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1595)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:888)
   [junit4]    > Caused by: org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:57344/wb_dlc/o
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:585)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
   [junit4]    > 	... 7 more
   [junit4]    > Caused by: java.net.SocketTimeoutException: Read timed out
   [junit4]    > 	at java.net.SocketInputStream.socketRead0(Native Method)
   [junit4]    > 	at java.net.SocketInputStream.read(SocketInputStream.java:152)
   [junit4]    > 	at java.net.SocketInputStream.read(SocketInputStream.java:122)
   [junit4]    > 	at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:160)
   [junit4]    > 	at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:84)
   [junit4]    > 	at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:273)
   [junit4]    > 	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
   [junit4]    > 	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
   [junit4]    > 	at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
   [junit4]    > 	at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283)
   [junit4]    > 	at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251)
   [junit4]    > 	at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197)
   [junit4]    > 	at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)
   [junit4]    > 	at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)
   [junit4]    > 	at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685)
   [junit4]    > 	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487)
   [junit4]    > 	at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
   [junit4]    > 	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
   [junit4]    > 	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
   [junit4]    > 	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:480)
   [junit4]    > 	... 11 moreThrowable #8: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=9598, name=SyncThread:0, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    > Caused by: java.lang.SecurityException: java.lang.System#exit(11) calls are not allowed because they terminate the test runner's JVM.
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([DC6E135E9ED086E8]:0)
   [junit4]    > 	at org.apache.lucene.util.TestSecurityManager$1.run(TestSecurityManager.java:86)
   [junit4]    > 	at org.apache.lucene.util.TestSecurityManager$1.run(TestSecurityManager.java:53)
   [junit4]    > 	at java.security.AccessController.doPrivileged(Native Method)
   [junit4]    > 	at org.apache.lucene.util.TestSecurityManager.checkExit(TestSecurityManager.java:53)
   [junit4]    > 	at java.lang.Runtime.exit(Runtime.java:107)
   [junit4]    > 	at java.lang.System.exit(System.java:962)
   [junit4]    > 	at org.apache.zookeeper.server.SyncRequestProcessor.run(SyncRequestProcessor.java:185)
   [junit4]   2> 6611272 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[DC6E135E9ED086E8]-worker) [n:127.0.0.1:33567_wb_dlc%2Fo c:awholynewstresscollection_collection0_1 s:shard7 r:core_node18 x:awholynewstresscollection_collection0_1_shard7_replica3] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_DC6E135E9ED086E8-001
   [junit4]   2> Oct 27, 2015 8:53:38 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene54): {other_tl1=BlockTreeOrds(blocksize=128), multiDefault=PostingsFormat(name=Direct), range_facet_i_dv=PostingsFormat(name=Direct), range_facet_l_dv=Lucene50(blocksize=128), a_t=PostingsFormat(name=Direct), _version_=BlockTreeOrds(blocksize=128), text=PostingsFormat(name=LuceneVarGapDocFreqInterval), intDefault=BlockTreeOrds(blocksize=128), a_i1=PostingsFormat(name=Direct), timestamp=PostingsFormat(name=Direct), id=Lucene50(blocksize=128), range_facet_l=PostingsFormat(name=Direct)}, docValues:{range_facet_i_dv=DocValuesFormat(name=Lucene54), range_facet_l_dv=DocValuesFormat(name=Memory), timestamp=DocValuesFormat(name=Lucene54)}, sim=RandomSimilarityProvider(queryNorm=false,coord=crazy): {}, locale=th, timezone=Turkey
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_72 (64-bit)/cpus=4,threads=1,free=168141152,total=493355008
   [junit4]   2> NOTE: All tests run in this JVM: [RollingRestartTest, TestSolr4Spatial2, SpatialFilterTest, TestUpdate, AddBlockUpdateTest, TestConfigOverlay, AsyncCallRequestStatusResponseTest, TestHighlightDedupGrouping, DistributedFacetPivotLongTailTest, TestManagedSchemaFieldTypeResource, PKIAuthenticationIntegrationTest, TestConfigSetProperties, TestRuleBasedAuthorizationPlugin, TestFastOutputStream, MinimalSchemaTest, ChaosMonkeySafeLeaderTest, NotRequiredUniqueKeyTest, TestDistributedGrouping, SegmentsInfoRequestHandlerTest, UniqFieldsUpdateProcessorFactoryTest, CollectionStateFormat2Test, BJQParserTest, BigEndianAscendingWordSerializerTest, SaslZkACLProviderTest, TestReloadDeadlock, TestBlobHandler, SpellPossibilityIteratorTest, CollectionsAPIDistributedZkTest]
   [junit4] Completed [480/543] on J1 in 5182.97s, 1 test, 1 error <<< FAILURES!

[...truncated 198 lines...]
   [junit4] JVM J1: stdout was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J1-20151027_190323_791.sysout
   [junit4] >>> JVM J1: stdout (verbatim) ----
   [junit4] java.lang.OutOfMemoryError: GC overhead limit exceeded
   [junit4] Dumping heap to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/heapdumps/java_pid18207.hprof ...
   [junit4] Heap dump file created [675873746 bytes in 6.622 secs]
   [junit4] <<< JVM J1: EOF ----

[...truncated 11 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 543 suites (7 ignored), 2146 tests, 2 errors, 31 ignored (18 assumptions) [seed: DC6E135E9ED086E8]

Total time: 260 minutes 33 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 998 - Still Failing

[Apache Jenkins Server <je...@builds.apache.org>]

Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/998/

2 tests failed.
FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=15048, name=collection3, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=15048, name=collection3, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:60778: collection already exists: awholynewstresscollection_collection3_4
	at __randomizedtesting.SeedInfo.seed([463320E19318F710]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1574)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1595)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:888)


FAILED:  org.apache.solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR

Error Message:
Captured an uncaught exception in thread: Thread[id=27952, name=coreZkRegister-2805-thread-2, state=RUNNABLE, group=TGRP-LeaderInitiatedRecoveryOnShardRestartTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=27952, name=coreZkRegister-2805-thread-2, state=RUNNABLE, group=TGRP-LeaderInitiatedRecoveryOnShardRestartTest]
Caused by: java.lang.AssertionError
	at __randomizedtesting.SeedInfo.seed([463320E19318F710]:0)
	at org.apache.solr.cloud.ZkController.updateLeaderInitiatedRecoveryState(ZkController.java:2133)
	at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:434)
	at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)
	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)
	at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:346)
	at org.apache.solr.cloud.ZkController.joinElection(ZkController.java:1113)
	at org.apache.solr.cloud.ZkController.register(ZkController.java:926)
	at org.apache.solr.cloud.ZkController.register(ZkController.java:881)
	at org.apache.solr.core.ZkContainer$2.run(ZkContainer.java:183)
	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 10459 lines...]
   [junit4] Suite: org.apache.solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_463320E19318F710-001/init-core-data-001
   [junit4]   2> 1042169 INFO  (SUITE-LeaderInitiatedRecoveryOnShardRestartTest-seed#[463320E19318F710]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   2> 1042175 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 1042177 INFO  (Thread-22791) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1042177 INFO  (Thread-22791) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 1042277 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.ZkTestServer start zk server on port:60764
   [junit4]   2> 1042277 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1042278 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1042280 INFO  (zkCallback-1481-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7a775add name:ZooKeeperConnection Watcher:127.0.0.1:60764 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1042280 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1042280 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1042280 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 1042291 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1042291 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1042294 INFO  (zkCallback-1482-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1f6dd026 name:ZooKeeperConnection Watcher:127.0.0.1:60764/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1042294 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1042294 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1042294 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 1042297 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 1042298 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 1042302 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 1042310 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 1042310 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 1042315 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 1042316 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 1042320 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1042320 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1042332 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 1042332 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 1042339 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 1042339 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 1042343 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 1042343 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 1042349 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 1042350 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 1042353 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1042353 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1042360 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1042360 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1042364 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 1042364 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 1042368 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 1042368 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 1042910 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_463320E19318F710-001/control-001/cores/collection1
   [junit4]   2> 1042915 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 1042927 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@4aeb148e{/,null,AVAILABLE}
   [junit4]   2> 1042927 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.e.j.s.ServerConnector Started ServerConnector@f929d0f{HTTP/1.1}{127.0.0.1:36796}
   [junit4]   2> 1042927 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.e.j.s.Server Started @1045824ms
   [junit4]   2> 1042928 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_463320E19318F710-001/control-001/cores, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_463320E19318F710-001/tempDir-001/control/data, hostContext=/, hostPort=36796}
   [junit4]   2> 1042928 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@6b482747
   [junit4]   2> 1042928 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_463320E19318F710-001/control-001/'
   [junit4]   2> 1042928 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1042928 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1042952 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1042957 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1042959 INFO  (zkCallback-1483-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@239fc541 name:ZooKeeperConnection Watcher:127.0.0.1:60764/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1042959 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1042960 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1042963 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1042963 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_463320E19318F710-001/control-001/solr.xml
   [junit4]   2> 1042994 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_463320E19318F710-001/control-001/cores
   [junit4]   2> 1042995 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.CoreContainer New CoreContainer 642035676
   [junit4]   2> 1042995 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_463320E19318F710-001/control-001/]
   [junit4]   2> 1042995 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_463320E19318F710-001/control-001/lib
   [junit4]   2> 1042995 WARN  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_463320E19318F710-001/control-001/lib).
   [junit4]   2> 1043017 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1043020 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1043021 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1043021 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1043023 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:60764/solr
   [junit4]   2> 1043023 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 1043023 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1043024 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1043027 INFO  (zkCallback-1485-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6d0f2475 name:ZooKeeperConnection Watcher:127.0.0.1:60764 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1043030 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1043031 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1043035 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [n:127.0.0.1:36796_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1043038 INFO  (zkCallback-1486-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@74af7f8c name:ZooKeeperConnection Watcher:127.0.0.1:60764/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1043038 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [n:127.0.0.1:36796_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1043039 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [n:127.0.0.1:36796_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 1043043 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [n:127.0.0.1:36796_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 1043057 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [n:127.0.0.1:36796_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 1043059 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [n:127.0.0.1:36796_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 1043063 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [n:127.0.0.1:36796_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 1043066 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [n:127.0.0.1:36796_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 1043069 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [n:127.0.0.1:36796_    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 1043070 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [n:127.0.0.1:36796_    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 1043075 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [n:127.0.0.1:36796_    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 1043077 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [n:127.0.0.1:36796_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:36796_
   [junit4]   2> 1043078 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [n:127.0.0.1:36796_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:36796_
   [junit4]   2> 1043080 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [n:127.0.0.1:36796_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 1043081 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [n:127.0.0.1:36796_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 1043083 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [n:127.0.0.1:36796_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1043084 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [n:127.0.0.1:36796_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94758470776324100-127.0.0.1:36796_-n_0000000000
   [junit4]   2> 1043084 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [n:127.0.0.1:36796_    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:36796_
   [junit4]   2> 1043084 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [n:127.0.0.1:36796_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 1043087 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [n:127.0.0.1:36796_    ] o.a.s.c.Overseer Overseer (id=94758470776324100-127.0.0.1:36796_-n_0000000000) starting
   [junit4]   2> 1043090 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [n:127.0.0.1:36796_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 1043105 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [n:127.0.0.1:36796_    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 1043105 INFO  (OverseerCollectionConfigSetProcessor-94758470776324100-127.0.0.1:36796_-n_0000000000) [n:127.0.0.1:36796_    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 1043105 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [n:127.0.0.1:36796_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1043108 INFO  (OverseerStateUpdate-94758470776324100-127.0.0.1:36796_-n_0000000000) [n:127.0.0.1:36796_    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 1043135 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [n:127.0.0.1:36796_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1043135 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [n:127.0.0.1:36796_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1043138 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [n:127.0.0.1:36796_    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_463320E19318F710-001/control-001/cores
   [junit4]   2> 1043139 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [n:127.0.0.1:36796_    ] o.a.s.c.SolrCore Created CoreDescriptor: {coreNodeName=, configSetProperties=configsetprops.json, shard=, transient=false, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_463320E19318F710-001/control-001/cores/collection1/, loadOnStartup=true, collection=control_collection, dataDir=data/, name=collection1, config=solrconfig.xml, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_463320E19318F710-001/control-001/cores/collection1, schema=schema.xml}
   [junit4]   2> 1043139 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [n:127.0.0.1:36796_    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_463320E19318F710-001/control-001/cores/collection1/
   [junit4]   2> 1043139 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [n:127.0.0.1:36796_    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 1043144 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [n:127.0.0.1:36796_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 1043144 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [n:127.0.0.1:36796_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1043146 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 1043146 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1043151 INFO  (OverseerStateUpdate-94758470776324100-127.0.0.1:36796_-n_0000000000) [n:127.0.0.1:36796_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "shard":null,
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "node_name":"127.0.0.1:36796_",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:36796",
   [junit4]   2>   "core":"collection1"} current state version: 0
   [junit4]   2> 1043152 INFO  (OverseerStateUpdate-94758470776324100-127.0.0.1:36796_-n_0000000000) [n:127.0.0.1:36796_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "shard":null,
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "node_name":"127.0.0.1:36796_",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:36796",
   [junit4]   2>   "core":"collection1"}
   [junit4]   2> 1043152 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 1043152 INFO  (OverseerStateUpdate-94758470776324100-127.0.0.1:36796_-n_0000000000) [n:127.0.0.1:36796_    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 1043153 INFO  (OverseerStateUpdate-94758470776324100-127.0.0.1:36796_-n_0000000000) [n:127.0.0.1:36796_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 1043156 INFO  (zkCallback-1486-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1044153 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1044153 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 1044153 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 1044154 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 1044158 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1044158 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_463320E19318F710-001/control-001/cores/collection1/'
   [junit4]   2> 1044158 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1044158 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1044193 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 1044202 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 1044258 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 1044295 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1044298 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 1044308 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1045882 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 1045895 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 1045901 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 1045919 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1045922 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1045926 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1045927 WARN  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 1045927 WARN  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 1045928 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1045928 WARN  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 1045929 WARN  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 1045930 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 1045930 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 1045930 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 1045930 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_463320E19318F710-001/control-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 1045931 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@4ee6be14
   [junit4]   2> 1045932 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_463320E19318F710-001/control-001/cores/collection1/data
   [junit4]   2> 1045932 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_463320E19318F710-001/control-001/cores/collection1/data/index/
   [junit4]   2> 1045932 WARN  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_463320E19318F710-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 1045933 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_463320E19318F710-001/control-001/cores/collection1/data/index
   [junit4]   2> 1045933 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=9, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=0.28515625, noCFSRatio=0.42100909561013533]
   [junit4]   2> 1045933 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@74d03d00 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@135506ea),segFN=segments_1,generation=1}
   [junit4]   2> 1045933 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 1045944 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 1045944 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 1045944 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 1045944 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1045944 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1045944 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 1045945 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1045945 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1045945 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 1045961 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1045963 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1045964 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_463320E19318F710-001/control-001/cores/collection1/data/
   [junit4]   2> 1045965 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_463320E19318F710-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 1045967 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1045989 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1046002 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /schema,/admin/luke,/admin/logging,/admin/plugins,/admin/properties,/config,/update/json/docs,/admin/system,/replication,/update/csv,/admin/threads,/update/json,standard,/admin/file,/admin/segments,/admin/mbeans,/get,/update,/admin/ping
   [junit4]   2> 1046017 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 1046018 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1046018 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1046019 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 1046019 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 1046020 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=34, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
   [junit4]   2> 1046020 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@74d03d00 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@135506ea),segFN=segments_1,generation=1}
   [junit4]   2> 1046020 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 1046020 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@544c7732[collection1] main
   [junit4]   2> 1046021 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 1046021 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1046021 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 1046022 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 1046022 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 1046022 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1046023 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 1046024 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1046024 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 1046025 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1046025 INFO  (searcherExecutor-2757-thread-1-processing-s:shard1 n:127.0.0.1:36796_ c:control_collection r:core_node1 x:collection1) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@544c7732[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1046028 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 1046028 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 1046028 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 1046029 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1516135536459251712
   [junit4]   2> 1046031 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 2.0ms to seed version buckets with highest version 1516135536459251712
   [junit4]   2> 1046031 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 1046033 INFO  (coreLoadExecutor-2756-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 1046033 INFO  (coreZkRegister-2750-thread-1-processing-s:shard1 n:127.0.0.1:36796_ c:control_collection r:core_node1 x:collection1) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:36796 collection:control_collection shard:shard1
   [junit4]   2> 1046034 INFO  (coreZkRegister-2750-thread-1-processing-s:shard1 n:127.0.0.1:36796_ c:control_collection r:core_node1 x:collection1) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 1046039 INFO  (coreZkRegister-2750-thread-1-processing-s:shard1 n:127.0.0.1:36796_ c:control_collection r:core_node1 x:collection1) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94758470776324100-core_node1-n_0000000000
   [junit4]   2> 1046039 INFO  (coreZkRegister-2750-thread-1-processing-s:shard1 n:127.0.0.1:36796_ c:control_collection r:core_node1 x:collection1) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 1046041 INFO  (coreZkRegister-2750-thread-1-processing-s:shard1 n:127.0.0.1:36796_ c:control_collection r:core_node1 x:collection1) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1046041 INFO  (coreZkRegister-2750-thread-1-processing-s:shard1 n:127.0.0.1:36796_ c:control_collection r:core_node1 x:collection1) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1046041 INFO  (coreZkRegister-2750-thread-1-processing-s:shard1 n:127.0.0.1:36796_ c:control_collection r:core_node1 x:collection1) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:36796/collection1/
   [junit4]   2> 1046041 INFO  (coreZkRegister-2750-thread-1-processing-s:shard1 n:127.0.0.1:36796_ c:control_collection r:core_node1 x:collection1) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 1046042 INFO  (coreZkRegister-2750-thread-1-processing-s:shard1 n:127.0.0.1:36796_ c:control_collection r:core_node1 x:collection1) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:36796/collection1/ has no replicas
   [junit4]   2> 1046042 INFO  (coreZkRegister-2750-thread-1-processing-s:shard1 n:127.0.0.1:36796_ c:control_collection r:core_node1 x:collection1) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:36796/collection1/ shard1
   [junit4]   2> 1046041 INFO  (OverseerStateUpdate-94758470776324100-127.0.0.1:36796_-n_0000000000) [n:127.0.0.1:36796_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 1046045 INFO  (coreZkRegister-2750-thread-1-processing-s:shard1 n:127.0.0.1:36796_ c:control_collection r:core_node1 x:collection1) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 1046049 INFO  (zkCallback-1486-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1046056 INFO  (coreZkRegister-2750-thread-1-processing-s:shard1 n:127.0.0.1:36796_ c:control_collection r:core_node1 x:collection1) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 1046064 INFO  (OverseerStateUpdate-94758470776324100-127.0.0.1:36796_-n_0000000000) [n:127.0.0.1:36796_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:36796",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 2
   [junit4]   2> 1046173 INFO  (zkCallback-1486-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1046217 INFO  (coreZkRegister-2750-thread-1-processing-s:shard1 n:127.0.0.1:36796_ c:control_collection r:core_node1 x:collection1) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:36796/collection1/ and leader is http://127.0.0.1:36796/collection1/
   [junit4]   2> 1046217 INFO  (coreZkRegister-2750-thread-1-processing-s:shard1 n:127.0.0.1:36796_ c:control_collection r:core_node1 x:collection1) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:36796
   [junit4]   2> 1046217 INFO  (coreZkRegister-2750-thread-1-processing-s:shard1 n:127.0.0.1:36796_ c:control_collection r:core_node1 x:collection1) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 1046217 INFO  (coreZkRegister-2750-thread-1-processing-s:shard1 n:127.0.0.1:36796_ c:control_collection r:core_node1 x:collection1) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 1046217 INFO  (coreZkRegister-2750-thread-1-processing-s:shard1 n:127.0.0.1:36796_ c:control_collection r:core_node1 x:collection1) [n:127.0.0.1:36796_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1046220 INFO  (OverseerStateUpdate-94758470776324100-127.0.0.1:36796_-n_0000000000) [n:127.0.0.1:36796_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "node_name":"127.0.0.1:36796_",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:36796",
   [junit4]   2>   "core":"collection1"} current state version: 3
   [junit4]   2> 1046221 INFO  (OverseerStateUpdate-94758470776324100-127.0.0.1:36796_-n_0000000000) [n:127.0.0.1:36796_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "node_name":"127.0.0.1:36796_",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:36796",
   [junit4]   2>   "core":"collection1"}
   [junit4]   2> 1046233 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1046235 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1046241 INFO  (zkCallback-1488-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@26670646 name:ZooKeeperConnection Watcher:127.0.0.1:60764/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1046243 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1046243 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1046243 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1046247 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 1046247 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with stateFormat=2
   [junit4]   2> 1046247 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1046255 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1046281 INFO  (zkCallback-1489-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6add3c50 name:ZooKeeperConnection Watcher:127.0.0.1:60764/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1046282 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1046282 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1046284 INFO  (OverseerStateUpdate-94758470776324100-127.0.0.1:36796_-n_0000000000) [n:127.0.0.1:36796_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"create",
   [junit4]   2>   "name":"collection1",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "stateFormat":"2"} current state version: 3
   [junit4]   2> 1046284 INFO  (OverseerStateUpdate-94758470776324100-127.0.0.1:36796_-n_0000000000) [n:127.0.0.1:36796_    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 1046285 INFO  (zkCallback-1486-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1046285 INFO  (zkCallback-1488-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1046394 INFO  (OverseerStateUpdate-94758470776324100-127.0.0.1:36796_-n_0000000000) [n:127.0.0.1:36796_    ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/collection1/state.json
   [junit4]   2> 1046789 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_463320E19318F710-001/shard-1-001/cores/collection1
   [junit4]   2> 1046790 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_463320E19318F710-001/shard-1-001
   [junit4]   2> 1046802 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 1046814 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@10e2510{/,null,AVAILABLE}
   [junit4]   2> 1046814 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.e.j.s.ServerConnector Started ServerConnector@2e3a0cd4{HTTP/1.1}{127.0.0.1:41209}
   [junit4]   2> 1046814 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.e.j.s.Server Started @1049711ms
   [junit4]   2> 1046815 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_463320E19318F710-001/tempDir-001/jetty1, hostPort=41209, solrconfig=solrconfig.xml, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_463320E19318F710-001/shard-1-001/cores}
   [junit4]   2> 1046815 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@6b482747
   [junit4]   2> 1046815 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_463320E19318F710-001/shard-1-001/'
   [junit4]   2> 1046815 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1046815 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1046843 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1046845 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1046849 INFO  (zkCallback-1490-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@78e433dc name:ZooKeeperConnection Watcher:127.0.0.1:60764/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1046853 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1046860 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1046874 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1046876 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_463320E19318F710-001/shard-1-001/solr.xml
   [junit4]   2> 1046901 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_463320E19318F710-001/shard-1-001/cores
   [junit4]   2> 1046903 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.CoreContainer New CoreContainer 893896919
   [junit4]   2> 1046903 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_463320E19318F710-001/shard-1-001/]
   [junit4]   2> 1046903 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_463320E19318F710-001/shard-1-001/lib
   [junit4]   2> 1046903 WARN  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_463320E19318F710-001/shard-1-001/lib).
   [junit4]   2> 1046915 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1046917 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1046917 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1046917 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1046918 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:60764/solr
   [junit4]   2> 1046918 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 1046918 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1046919 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1046921 INFO  (zkCallback-1492-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@da0cfce name:ZooKeeperConnection Watcher:127.0.0.1:60764 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1046923 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1046923 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1046927 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [n:127.0.0.1:41209_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1046928 INFO  (zkCallback-1493-thread-1-processing-n:127.0.0.1:41209_) [n:127.0.0.1:41209_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7d1d4b72 name:ZooKeeperConnection Watcher:127.0.0.1:60764/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1046928 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [n:127.0.0.1:41209_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1046950 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [n:127.0.0.1:41209_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1047963 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [n:127.0.0.1:41209_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:41209_
   [junit4]   2> 1047964 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [n:127.0.0.1:41209_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:41209_
   [junit4]   2> 1047965 INFO  (zkCallback-1486-thread-1-processing-n:127.0.0.1:36796_) [n:127.0.0.1:36796_    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1047965 INFO  (zkCallback-1488-thread-1) [    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1047966 INFO  (zkCallback-1493-thread-1-processing-n:127.0.0.1:41209_) [n:127.0.0.1:41209_    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1047968 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [n:127.0.0.1:41209_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1047968 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [n:127.0.0.1:41209_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94758470776324105-127.0.0.1:41209_-n_0000000001
   [junit4]   2> 1047969 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [n:127.0.0.1:41209_    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/94758470776324100-127.0.0.1:36796_-n_0000000000 to know if I could be the leader
   [junit4]   2> 1048016 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [n:127.0.0.1:41209_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1048016 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [n:127.0.0.1:41209_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1048023 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [n:127.0.0.1:41209_    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_463320E19318F710-001/shard-1-001/cores
   [junit4]   2> 1048024 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [n:127.0.0.1:41209_    ] o.a.s.c.SolrCore Created CoreDescriptor: {config=solrconfig.xml, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_463320E19318F710-001/shard-1-001/cores/collection1/, coreNodeName=, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_463320E19318F710-001/shard-1-001/cores/collection1, loadOnStartup=true, configSetProperties=configsetprops.json, collection=collection1, schema=schema.xml, dataDir=data/, shard=, name=collection1, transient=false}
   [junit4]   2> 1048024 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [n:127.0.0.1:41209_    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_463320E19318F710-001/shard-1-001/cores/collection1/
   [junit4]   2> 1048024 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [n:127.0.0.1:41209_    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 1048027 INFO  (coreLoadExecutor-2767-thread-1-processing-n:127.0.0.1:41209_) [n:127.0.0.1:41209_ c:collection1   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 1048027 INFO  (coreLoadExecutor-2767-thread-1-processing-n:127.0.0.1:41209_) [n:127.0.0.1:41209_ c:collection1   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1048031 INFO  (OverseerStateUpdate-94758470776324100-127.0.0.1:36796_-n_0000000000) [n:127.0.0.1:36796_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "shard":null,
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "node_name":"127.0.0.1:41209_",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "base_url":"http://127.0.0.1:41209",
   [junit4]   2>   "core":"collection1"} current state version: 4
   [junit4]   2> 1048032 INFO  (OverseerStateUpdate-94758470776324100-127.0.0.1:36796_-n_0000000000) [n:127.0.0.1:36796_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "shard":null,
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "node_name":"127.0.0.1:41209_",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "base_url":"http://127.0.0.1:41209",
   [junit4]   2>   "core":"collection1"}
   [junit4]   2> 1048032 INFO  (OverseerStateUpdate-94758470776324100-127.0.0.1:36796_-n_0000000000) [n:127.0.0.1:36796_    ] o.a.s.c.o.ReplicaMutator Collection already exists with numShards=1
   [junit4]   2> 1048032 INFO  (OverseerStateUpdate-94758470776324100-127.0.0.1:36796_-n_0000000000) [n:127.0.0.1:36796_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 1048034 INFO  (OverseerStateUpdate-94758470776324100-127.0.0.1:36796_-n_0000000000) [n:127.0.0.1:36796_    ] o.a.s.c.o.ZkStateWriter going to update_collection /collections/collection1/state.json version: 0
   [junit4]   2> 1048034 INFO  (coreLoadExecutor-2767-thread-1-processing-n:127.0.0.1:41209_) [n:127.0.0.1:41209_ c:collection1   x:collection1] o.a.s.c.ZkController Registering watch for collection collection1
   [junit4]   2> 1048035 INFO  (coreLoadExecutor-2767-thread-1-processing-n:127.0.0.1:41209_) [n:127.0.0.1:41209_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader addZkWatch collection1
   [junit4]   2> 1048039 INFO  (coreLoadExecutor-2767-thread-1-processing-n:127.0.0.1:41209_) [n:127.0.0.1:41209_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Add data for collection1 ver 1 
   [junit4]   2> 1048039 INFO  (coreLoadExecutor-2767-thread-1-processing-n:127.0.0.1:41209_) [n:127.0.0.1:41209_ c:collection1   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 1048039 INFO  (coreLoadExecutor-2767-thread-1-processing-n:127.0.0.1:41209_) [n:127.0.0.1:41209_ c:collection1   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1048039 INFO  (coreLoadExecutor-2767-thread-1-processing-n:127.0.0.1:41209_) [n:127.0.0.1:41209_ c:collection1   x:collection1] o.a.s.c.ZkController Check for collection zkNode:collection1
   [junit4]   2> 1048040 INFO  (coreLoadExecutor-2767-thread-1-processing-n:127.0.0.1:41209_) [n:127.0.0.1:41209_ c:collection1   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 1048040 INFO  (coreLoadExecutor-2767-thread-1-processing-n:127.0.0.1:41209_) [n:127.0.0.1:41209_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/collection1
   [junit4]   2> 1048041 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [n:127.0.0.1:41209_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 1048041 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[463320E19318F710]) [n:127.0.0.1:41209_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1048043 INFO  (coreLoadExecutor-2767-thread-1-processing-n:127.0.0.1:41209_) [n:127.0.0.1:41209_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1048043 INFO  (coreLoadExecutor-2767-thread-1-processing-n:127.0.0.1:41209_) [n:127.0.0.1:41209_ c:

[...truncated too long message...]

.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:888)Throwable #4: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=15046, name=collection1, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    > Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:60778: collection already exists: awholynewstresscollection_collection1_4
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([463320E19318F710]:0)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1574)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1595)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:888)Throwable #5: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=15047, name=collection2, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    > Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:60778: collection already exists: awholynewstresscollection_collection2_4
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([463320E19318F710]:0)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1574)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1595)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:888)Throwable #6: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=15050, name=collection5, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    > Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:60778: Could not find collection : awholynewstresscollection_collection5_4
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([463320E19318F710]:0)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:895)
   [junit4]   2> 1902553 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[463320E19318F710]-worker) [n:127.0.0.1:57793_ c:addReplicaColl s:shard2 r:core_node6 x:addReplicaColl_shard2_replica3] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene53): {intDefault=PostingsFormat(name=LuceneFixedGap), range_facet_i_dv=PostingsFormat(name=LuceneFixedGap), other_tl1=PostingsFormat(name=LuceneFixedGap), a_t=PostingsFormat(name=LuceneFixedGap), _version_=PostingsFormat(name=LuceneFixedGap), a_i1=PostingsFormat(name=LuceneFixedGap), range_facet_l_dv=PostingsFormat(name=LuceneVarGapFixedInterval), multiDefault=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), text=Lucene50(blocksize=128), range_facet_l=PostingsFormat(name=LuceneFixedGap), id=PostingsFormat(name=LuceneVarGapFixedInterval), timestamp=PostingsFormat(name=LuceneFixedGap)}, docValues:{range_facet_i_dv=DocValuesFormat(name=Memory), range_facet_l_dv=DocValuesFormat(name=Direct), timestamp=DocValuesFormat(name=Memory)}, sim=RandomSimilarityProvider(queryNorm=false,coord=yes): {}, locale=zh_TW, timezone=America/North_Dakota/Center
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_72 (64-bit)/cpus=4,threads=1,free=72746112,total=508559360
   [junit4]   2> NOTE: All tests run in this JVM: [AsyncCallRequestStatusResponseTest, IgnoreCommitOptimizeUpdateProcessorFactoryTest, TestRestoreCore, BasicFunctionalityTest, TestStandardQParsers, HdfsCollectionsAPIDistributedZkTest, SliceStateTest, TestDistributedGrouping, TestSuggestSpellingConverter, CustomCollectionTest, PreAnalyzedFieldTest, AddSchemaFieldsUpdateProcessorFactoryTest, TestHdfsUpdateLog, TestManagedSchemaDynamicFieldResource, CircularListTest, DocumentBuilderTest, LoggingHandlerTest, TestSolr4Spatial2, TestImplicitCoreProperties, DistributedFacetPivotLongTailTest, DocValuesTest, SaslZkACLProviderTest, OverseerTaskQueueTest, SpellCheckComponentTest, CursorPagingTest, TestSortingResponseWriter, TestSolrCoreProperties, TestQueryTypes, HdfsNNFailoverTest, SimplePostToolTest, DOMUtilTest, HdfsSyncSliceTest, AnalyticsQueryTest, ConvertedLegacyTest, SolrPluginUtilsTest, TestNoOpRegenerator, MoreLikeThisHandlerTest, TestFastOutputStream, TestCrossCoreJoin, SpellCheckCollatorTest, OverseerStatusTest, BJQParserTest, TestFieldTypeResource, TermsComponentTest, TestCodecSupport, AtomicUpdatesTest, DistributedTermsComponentTest, TestStressVersions, SolrCmdDistributorTest, TestSolrDeletionPolicy2, TestUtils, TestJmxMonitoredMap, TestClusterStateMutator, RecoveryAfterSoftCommitTest, SuggesterFSTTest, TestConfigOverlay, SolrIndexSplitterTest, TestBulkSchemaConcurrent, TestDFRSimilarityFactory, TestTrieFacet, TestRandomRequestDistribution, SuggestComponentContextFilterQueryTest, URLClassifyProcessorTest, OpenCloseCoreStressTest, TestHighlightDedupGrouping, RequestLoggingTest, TestQuerySenderNoQuery, TestAnalyzeInfixSuggestions, TestLeaderInitiatedRecoveryThread, TestWordDelimiterFilterFactory, TestManagedSchemaFieldTypeResource, TestDefaultStatsCache, FastVectorHighlighterTest, TestRestManager, CollectionsAPIDistributedZkTest]
   [junit4] Completed [222/543] on J1 in 363.93s, 1 test, 1 error <<< FAILURES!

[...truncated 1059 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 543 suites (7 ignored), 2144 tests, 2 errors, 31 ignored (18 assumptions) [seed: 463320E19318F710]

Total time: 227 minutes 55 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 997 - Still Failing

[Apache Jenkins Server <je...@builds.apache.org>]

Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/997/

2 tests failed.
FAILED:  org.apache.solr.handler.TestReplicationHandler.doTestReplicateAfterCoreReload

Error Message:
expected:<[{indexVersion=1445818330639,generation=2,filelist=[_2d.fdt, _2d.fdx, _2d.fnm, _2d.nvd, _2d.nvm, _2d.si, _2d_Asserting_0.doc, _2d_Asserting_0.tim, _2d_Asserting_0.tip, _4p.fdt, _4p.fdx, _4p.fnm, _4p.nvd, _4p.nvm, _4p.si, _4p_Asserting_0.doc, _4p_Asserting_0.tim, _4p_Asserting_0.tip, _4q.fdt, _4q.fdx, _4q.fnm, _4q.nvd, _4q.nvm, _4q.si, _4q_Asserting_0.doc, _4q_Asserting_0.tim, _4q_Asserting_0.tip, _4r.fdt, _4r.fdx, _4r.fnm, _4r.nvd, _4r.nvm, _4r.si, _4r_Asserting_0.doc, _4r_Asserting_0.tim, _4r_Asserting_0.tip, _4s.fdt, _4s.fdx, _4s.fnm, _4s.nvd, _4s.nvm, _4s.si, _4s_Asserting_0.doc, _4s_Asserting_0.tim, _4s_Asserting_0.tip, _4t.fdt, _4t.fdx, _4t.fnm, _4t.nvd, _4t.nvm, _4t.si, _4t_Asserting_0.doc, _4t_Asserting_0.tim, _4t_Asserting_0.tip, _4u.fdt, _4u.fdx, _4u.fnm, _4u.nvd, _4u.nvm, _4u.si, _4u_Asserting_0.doc, _4u_Asserting_0.tim, _4u_Asserting_0.tip, _4v.fdt, _4v.fdx, _4v.fnm, _4v.nvd, _4v.nvm, _4v.si, _4v_Asserting_0.doc, _4v_Asserting_0.tim, _4v_Asserting_0.tip, _4w.fdt, _4w.fdx, _4w.fnm, _4w.nvd, _4w.nvm, _4w.si, _4w_Asserting_0.doc, _4w_Asserting_0.tim, _4w_Asserting_0.tip, _4x.fdt, _4x.fdx, _4x.fnm, _4x.nvd, _4x.nvm, _4x.si, _4x_Asserting_0.doc, _4x_Asserting_0.tim, _4x_Asserting_0.tip, _4y.fdt, _4y.fdx, _4y.fnm, _4y.nvd, _4y.nvm, _4y.si, _4y_Asserting_0.doc, _4y_Asserting_0.tim, _4y_Asserting_0.tip, _4z.fdt, _4z.fdx, _4z.fnm, _4z.nvd, _4z.nvm, _4z.si, _4z_Asserting_0.doc, _4z_Asserting_0.tim, _4z_Asserting_0.tip, _50.fdt, _50.fdx, _50.fnm, _50.nvd, _50.nvm, _50.si, _50_Asserting_0.doc, _50_Asserting_0.tim, _50_Asserting_0.tip, _51.fdt, _51.fdx, _51.fnm, _51.nvd, _51.nvm, _51.si, _51_Asserting_0.doc, _51_Asserting_0.tim, _51_Asserting_0.tip, _52.fdt, _52.fdx, _52.fnm, _52.nvd, _52.nvm, _52.si, _52_Asserting_0.doc, _52_Asserting_0.tim, _52_Asserting_0.tip, _53.fdt, _53.fdx, _53.fnm, _53.nvd, _53.nvm, _53.si, _53_Asserting_0.doc, _53_Asserting_0.tim, _53_Asserting_0.tip, _54.fdt, _54.fdx, _54.fnm, _54.nvd, _54.nvm, _54.si, _54_Asserting_0.doc, _54_Asserting_0.tim, _54_Asserting_0.tip, _55.fdt, _55.fdx, _55.fnm, _55.nvd, _55.nvm, _55.si, _55_Asserting_0.doc, _55_Asserting_0.tim, _55_Asserting_0.tip, _56.fdt, _56.fdx, _56.fnm, _56.nvd, _56.nvm, _56.si, _56_Asserting_0.doc, _56_Asserting_0.tim, _56_Asserting_0.tip, _57.fdt, _57.fdx, _57.fnm, _57.nvd, _57.nvm, _57.si, _57_Asserting_0.doc, _57_Asserting_0.tim, _57_Asserting_0.tip, _58.fdt, _58.fdx, _58.fnm, _58.nvd, _58.nvm, _58.si, _58_Asserting_0.doc, _58_Asserting_0.tim, _58_Asserting_0.tip, _59.fdt, _59.fdx, _59.fnm, _59.nvd, _59.nvm, _59.si, _59_Asserting_0.doc, _59_Asserting_0.tim, _59_Asserting_0.tip, _5a.fdt, _5a.fdx, _5a.fnm, _5a.nvd, _5a.nvm, _5a.si, _5a_Asserting_0.doc, _5a_Asserting_0.tim, _5a_Asserting_0.tip, _5b.fdt, _5b.fdx, _5b.fnm, _5b.nvd, _5b.nvm, _5b.si, _5b_Asserting_0.doc, _5b_Asserting_0.tim, _5b_Asserting_0.tip, _5c.fdt, _5c.fdx, _5c.fnm, _5c.nvd, _5c.nvm, _5c.si, _5c_Asserting_0.doc, _5c_Asserting_0.tim, _5c_Asserting_0.tip, _5d.fdt, _5d.fdx, _5d.fnm, _5d.nvd, _5d.nvm, _5d.si, _5d_Asserting_0.doc, _5d_Asserting_0.tim, _5d_Asserting_0.tip, _5e.fdt, _5e.fdx, _5e.fnm, _5e.nvd, _5e.nvm, _5e.si, _5e_Asserting_0.doc, _5e_Asserting_0.tim, _5e_Asserting_0.tip, _5f.fdt, _5f.fdx, _5f.fnm, _5f.nvd, _5f.nvm, _5f.si, _5f_Asserting_0.doc, _5f_Asserting_0.tim, _5f_Asserting_0.tip, _5g.fdt, _5g.fdx, _5g.fnm, _5g.nvd, _5g.nvm, _5g.si, _5g_Asserting_0.doc, _5g_Asserting_0.tim, _5g_Asserting_0.tip, _5h.fdt, _5h.fdx, _5h.fnm, _5h.nvd, _5h.nvm, _5h.si, _5h_Asserting_0.doc, _5h_Asserting_0.tim, _5h_Asserting_0.tip, _5i.fdt, _5i.fdx, _5i.fnm, _5i.nvd, _5i.nvm, _5i.si, _5i_Asserting_0.doc, _5i_Asserting_0.tim, _5i_Asserting_0.tip, _5j.fdt, _5j.fdx, _5j.fnm, _5j.nvd, _5j.nvm, _5j.si, _5j_Asserting_0.doc, _5j_Asserting_0.tim, _5j_Asserting_0.tip, _5k.fdt, _5k.fdx, _5k.fnm, _5k.nvd, _5k.nvm, _5k.si, _5k_Asserting_0.doc, _5k_Asserting_0.tim, _5k_Asserting_0.tip, _5l.fdt, _5l.fdx, _5l.fnm, _5l.nvd, _5l.nvm, _5l.si, _5l_Asserting_0.doc, _5l_Asserting_0.tim, _5l_Asserting_0.tip, _5m.fdt, _5m.fdx, _5m.fnm, _5m.nvd, _5m.nvm, _5m.si, _5m_Asserting_0.doc, _5m_Asserting_0.tim, _5m_Asserting_0.tip, _5n.fdt, _5n.fdx, _5n.fnm, _5n.nvd, _5n.nvm, _5n.si, _5n_Asserting_0.doc, _5n_Asserting_0.tim, _5n_Asserting_0.tip, _5o.fdt, _5o.fdx, _5o.fnm, _5o.nvd, _5o.nvm, _5o.si, _5o_Asserting_0.doc, _5o_Asserting_0.tim, _5o_Asserting_0.tip, _5p.fdt, _5p.fdx, _5p.fnm, _5p.nvd, _5p.nvm, _5p.si, _5p_Asserting_0.doc, _5p_Asserting_0.tim, _5p_Asserting_0.tip, _5q.fdt, _5q.fdx, _5q.fnm, _5q.nvd, _5q.nvm, _5q.si, _5q_Asserting_0.doc, _5q_Asserting_0.tim, _5q_Asserting_0.tip, _5r.fdt, _5r.fdx, _5r.fnm, _5r.nvd, _5r.nvm, _5r.si, _5r_Asserting_0.doc, _5r_Asserting_0.tim, _5r_Asserting_0.tip, _5s.fdt, _5s.fdx, _5s.fnm, _5s.nvd, _5s.nvm, _5s.si, _5s_Asserting_0.doc, _5s_Asserting_0.tim, _5s_Asserting_0.tip, _5t.fdt, _5t.fdx, _5t.fnm, _5t.nvd, _5t.nvm, _5t.si, _5t_Asserting_0.doc, _5t_Asserting_0.tim, _5t_Asserting_0.tip, _5u.fdt, _5u.fdx, _5u.fnm, _5u.nvd, _5u.nvm, _5u.si, _5u_Asserting_0.doc, _5u_Asserting_0.tim, _5u_Asserting_0.tip, _5v.fdt, _5v.fdx, _5v.fnm, _5v.nvd, _5v.nvm, _5v.si, _5v_Asserting_0.doc, _5v_Asserting_0.tim, _5v_Asserting_0.tip, segments_2]}]> but was:<[{indexVersion=1445818330639,generation=2,filelist=[_2d.fdt, _2d.fdx, _2d.fnm, _2d.nvd, _2d.nvm, _2d.si, _2d_Asserting_0.doc, _2d_Asserting_0.tim, _2d_Asserting_0.tip, _4p.fdt, _4p.fdx, _4p.fnm, _4p.nvd, _4p.nvm, _4p.si, _4p_Asserting_0.doc, _4p_Asserting_0.tim, _4p_Asserting_0.tip, _4q.fdt, _4q.fdx, _4q.fnm, _4q.nvd, _4q.nvm, _4q.si, _4q_Asserting_0.doc, _4q_Asserting_0.tim, _4q_Asserting_0.tip, _4r.fdt, _4r.fdx, _4r.fnm, _4r.nvd, _4r.nvm, _4r.si, _4r_Asserting_0.doc, _4r_Asserting_0.tim, _4r_Asserting_0.tip, _4s.fdt, _4s.fdx, _4s.fnm, _4s.nvd, _4s.nvm, _4s.si, _4s_Asserting_0.doc, _4s_Asserting_0.tim, _4s_Asserting_0.tip, _4t.fdt, _4t.fdx, _4t.fnm, _4t.nvd, _4t.nvm, _4t.si, _4t_Asserting_0.doc, _4t_Asserting_0.tim, _4t_Asserting_0.tip, _4u.fdt, _4u.fdx, _4u.fnm, _4u.nvd, _4u.nvm, _4u.si, _4u_Asserting_0.doc, _4u_Asserting_0.tim, _4u_Asserting_0.tip, _4v.fdt, _4v.fdx, _4v.fnm, _4v.nvd, _4v.nvm, _4v.si, _4v_Asserting_0.doc, _4v_Asserting_0.tim, _4v_Asserting_0.tip, _4w.fdt, _4w.fdx, _4w.fnm, _4w.nvd, _4w.nvm, _4w.si, _4w_Asserting_0.doc, _4w_Asserting_0.tim, _4w_Asserting_0.tip, _4x.fdt, _4x.fdx, _4x.fnm, _4x.nvd, _4x.nvm, _4x.si, _4x_Asserting_0.doc, _4x_Asserting_0.tim, _4x_Asserting_0.tip, _4y.fdt, _4y.fdx, _4y.fnm, _4y.nvd, _4y.nvm, _4y.si, _4y_Asserting_0.doc, _4y_Asserting_0.tim, _4y_Asserting_0.tip, _4z.fdt, _4z.fdx, _4z.fnm, _4z.nvd, _4z.nvm, _4z.si, _4z_Asserting_0.doc, _4z_Asserting_0.tim, _4z_Asserting_0.tip, _50.fdt, _50.fdx, _50.fnm, _50.nvd, _50.nvm, _50.si, _50_Asserting_0.doc, _50_Asserting_0.tim, _50_Asserting_0.tip, _51.fdt, _51.fdx, _51.fnm, _51.nvd, _51.nvm, _51.si, _51_Asserting_0.doc, _51_Asserting_0.tim, _51_Asserting_0.tip, _52.fdt, _52.fdx, _52.fnm, _52.nvd, _52.nvm, _52.si, _52_Asserting_0.doc, _52_Asserting_0.tim, _52_Asserting_0.tip, _53.fdt, _53.fdx, _53.fnm, _53.nvd, _53.nvm, _53.si, _53_Asserting_0.doc, _53_Asserting_0.tim, _53_Asserting_0.tip, _54.fdt, _54.fdx, _54.fnm, _54.nvd, _54.nvm, _54.si, _54_Asserting_0.doc, _54_Asserting_0.tim, _54_Asserting_0.tip, _55.fdt, _55.fdx, _55.fnm, _55.nvd, _55.nvm, _55.si, _55_Asserting_0.doc, _55_Asserting_0.tim, _55_Asserting_0.tip, _56.fdt, _56.fdx, _56.fnm, _56.nvd, _56.nvm, _56.si, _56_Asserting_0.doc, _56_Asserting_0.tim, _56_Asserting_0.tip, _57.fdt, _57.fdx, _57.fnm, _57.nvd, _57.nvm, _57.si, _57_Asserting_0.doc, _57_Asserting_0.tim, _57_Asserting_0.tip, _58.fdt, _58.fdx, _58.fnm, _58.nvd, _58.nvm, _58.si, _58_Asserting_0.doc, _58_Asserting_0.tim, _58_Asserting_0.tip, _59.fdt, _59.fdx, _59.fnm, _59.nvd, _59.nvm, _59.si, _59_Asserting_0.doc, _59_Asserting_0.tim, _59_Asserting_0.tip, _5a.fdt, _5a.fdx, _5a.fnm, _5a.nvd, _5a.nvm, _5a.si, _5a_Asserting_0.doc, _5a_Asserting_0.tim, _5a_Asserting_0.tip, _5b.fdt, _5b.fdx, _5b.fnm, _5b.nvd, _5b.nvm, _5b.si, _5b_Asserting_0.doc, _5b_Asserting_0.tim, _5b_Asserting_0.tip, _5c.fdt, _5c.fdx, _5c.fnm, _5c.nvd, _5c.nvm, _5c.si, _5c_Asserting_0.doc, _5c_Asserting_0.tim, _5c_Asserting_0.tip, _5d.fdt, _5d.fdx, _5d.fnm, _5d.nvd, _5d.nvm, _5d.si, _5d_Asserting_0.doc, _5d_Asserting_0.tim, _5d_Asserting_0.tip, _5e.fdt, _5e.fdx, _5e.fnm, _5e.nvd, _5e.nvm, _5e.si, _5e_Asserting_0.doc, _5e_Asserting_0.tim, _5e_Asserting_0.tip, _5f.fdt, _5f.fdx, _5f.fnm, _5f.nvd, _5f.nvm, _5f.si, _5f_Asserting_0.doc, _5f_Asserting_0.tim, _5f_Asserting_0.tip, _5g.fdt, _5g.fdx, _5g.fnm, _5g.nvd, _5g.nvm, _5g.si, _5g_Asserting_0.doc, _5g_Asserting_0.tim, _5g_Asserting_0.tip, _5h.fdt, _5h.fdx, _5h.fnm, _5h.nvd, _5h.nvm, _5h.si, _5h_Asserting_0.doc, _5h_Asserting_0.tim, _5h_Asserting_0.tip, _5i.fdt, _5i.fdx, _5i.fnm, _5i.nvd, _5i.nvm, _5i.si, _5i_Asserting_0.doc, _5i_Asserting_0.tim, _5i_Asserting_0.tip, _5j.fdt, _5j.fdx, _5j.fnm, _5j.nvd, _5j.nvm, _5j.si, _5j_Asserting_0.doc, _5j_Asserting_0.tim, _5j_Asserting_0.tip, _5k.fdt, _5k.fdx, _5k.fnm, _5k.nvd, _5k.nvm, _5k.si, _5k_Asserting_0.doc, _5k_Asserting_0.tim, _5k_Asserting_0.tip, _5l.fdt, _5l.fdx, _5l.fnm, _5l.nvd, _5l.nvm, _5l.si, _5l_Asserting_0.doc, _5l_Asserting_0.tim, _5l_Asserting_0.tip, _5m.fdt, _5m.fdx, _5m.fnm, _5m.nvd, _5m.nvm, _5m.si, _5m_Asserting_0.doc, _5m_Asserting_0.tim, _5m_Asserting_0.tip, _5n.fdt, _5n.fdx, _5n.fnm, _5n.nvd, _5n.nvm, _5n.si, _5n_Asserting_0.doc, _5n_Asserting_0.tim, _5n_Asserting_0.tip, _5o.fdt, _5o.fdx, _5o.fnm, _5o.nvd, _5o.nvm, _5o.si, _5o_Asserting_0.doc, _5o_Asserting_0.tim, _5o_Asserting_0.tip, _5p.fdt, _5p.fdx, _5p.fnm, _5p.nvd, _5p.nvm, _5p.si, _5p_Asserting_0.doc, _5p_Asserting_0.tim, _5p_Asserting_0.tip, _5q.fdt, _5q.fdx, _5q.fnm, _5q.nvd, _5q.nvm, _5q.si, _5q_Asserting_0.doc, _5q_Asserting_0.tim, _5q_Asserting_0.tip, _5r.fdt, _5r.fdx, _5r.fnm, _5r.nvd, _5r.nvm, _5r.si, _5r_Asserting_0.doc, _5r_Asserting_0.tim, _5r_Asserting_0.tip, _5s.fdt, _5s.fdx, _5s.fnm, _5s.nvd, _5s.nvm, _5s.si, _5s_Asserting_0.doc, _5s_Asserting_0.tim, _5s_Asserting_0.tip, _5t.fdt, _5t.fdx, _5t.fnm, _5t.nvd, _5t.nvm, _5t.si, _5t_Asserting_0.doc, _5t_Asserting_0.tim, _5t_Asserting_0.tip, _5u.fdt, _5u.fdx, _5u.fnm, _5u.nvd, _5u.nvm, _5u.si, _5u_Asserting_0.doc, _5u_Asserting_0.tim, _5u_Asserting_0.tip, _5v.fdt, _5v.fdx, _5v.fnm, _5v.nvd, _5v.nvm, _5v.si, _5v_Asserting_0.doc, _5v_Asserting_0.tim, _5v_Asserting_0.tip, segments_2]}, {indexVersion=1445818330639,generation=3,filelist=[_2d.fdt, _2d.fdx, _2d.fnm, _2d.nvd, _2d.nvm, _2d.si, _2d_Asserting_0.doc, _2d_Asserting_0.tim, _2d_Asserting_0.tip, _4q.fdt, _4q.fdx, _4q.fnm, _4q.nvd, _4q.nvm, _4q.si, _4q_Asserting_0.doc, _4q_Asserting_0.tim, _4q_Asserting_0.tip, _5w.cfe, _5w.cfs, _5w.si, segments_3]}]>

Stack Trace:
java.lang.AssertionError: expected:<[{indexVersion=1445818330639,generation=2,filelist=[_2d.fdt, _2d.fdx, _2d.fnm, _2d.nvd, _2d.nvm, _2d.si, _2d_Asserting_0.doc, _2d_Asserting_0.tim, _2d_Asserting_0.tip, _4p.fdt, _4p.fdx, _4p.fnm, _4p.nvd, _4p.nvm, _4p.si, _4p_Asserting_0.doc, _4p_Asserting_0.tim, _4p_Asserting_0.tip, _4q.fdt, _4q.fdx, _4q.fnm, _4q.nvd, _4q.nvm, _4q.si, _4q_Asserting_0.doc, _4q_Asserting_0.tim, _4q_Asserting_0.tip, _4r.fdt, _4r.fdx, _4r.fnm, _4r.nvd, _4r.nvm, _4r.si, _4r_Asserting_0.doc, _4r_Asserting_0.tim, _4r_Asserting_0.tip, _4s.fdt, _4s.fdx, _4s.fnm, _4s.nvd, _4s.nvm, _4s.si, _4s_Asserting_0.doc, _4s_Asserting_0.tim, _4s_Asserting_0.tip, _4t.fdt, _4t.fdx, _4t.fnm, _4t.nvd, _4t.nvm, _4t.si, _4t_Asserting_0.doc, _4t_Asserting_0.tim, _4t_Asserting_0.tip, _4u.fdt, _4u.fdx, _4u.fnm, _4u.nvd, _4u.nvm, _4u.si, _4u_Asserting_0.doc, _4u_Asserting_0.tim, _4u_Asserting_0.tip, _4v.fdt, _4v.fdx, _4v.fnm, _4v.nvd, _4v.nvm, _4v.si, _4v_Asserting_0.doc, _4v_Asserting_0.tim, _4v_Asserting_0.tip, _4w.fdt, _4w.fdx, _4w.fnm, _4w.nvd, _4w.nvm, _4w.si, _4w_Asserting_0.doc, _4w_Asserting_0.tim, _4w_Asserting_0.tip, _4x.fdt, _4x.fdx, _4x.fnm, _4x.nvd, _4x.nvm, _4x.si, _4x_Asserting_0.doc, _4x_Asserting_0.tim, _4x_Asserting_0.tip, _4y.fdt, _4y.fdx, _4y.fnm, _4y.nvd, _4y.nvm, _4y.si, _4y_Asserting_0.doc, _4y_Asserting_0.tim, _4y_Asserting_0.tip, _4z.fdt, _4z.fdx, _4z.fnm, _4z.nvd, _4z.nvm, _4z.si, _4z_Asserting_0.doc, _4z_Asserting_0.tim, _4z_Asserting_0.tip, _50.fdt, _50.fdx, _50.fnm, _50.nvd, _50.nvm, _50.si, _50_Asserting_0.doc, _50_Asserting_0.tim, _50_Asserting_0.tip, _51.fdt, _51.fdx, _51.fnm, _51.nvd, _51.nvm, _51.si, _51_Asserting_0.doc, _51_Asserting_0.tim, _51_Asserting_0.tip, _52.fdt, _52.fdx, _52.fnm, _52.nvd, _52.nvm, _52.si, _52_Asserting_0.doc, _52_Asserting_0.tim, _52_Asserting_0.tip, _53.fdt, _53.fdx, _53.fnm, _53.nvd, _53.nvm, _53.si, _53_Asserting_0.doc, _53_Asserting_0.tim, _53_Asserting_0.tip, _54.fdt, _54.fdx, _54.fnm, _54.nvd, _54.nvm, _54.si, _54_Asserting_0.doc, _54_Asserting_0.tim, _54_Asserting_0.tip, _55.fdt, _55.fdx, _55.fnm, _55.nvd, _55.nvm, _55.si, _55_Asserting_0.doc, _55_Asserting_0.tim, _55_Asserting_0.tip, _56.fdt, _56.fdx, _56.fnm, _56.nvd, _56.nvm, _56.si, _56_Asserting_0.doc, _56_Asserting_0.tim, _56_Asserting_0.tip, _57.fdt, _57.fdx, _57.fnm, _57.nvd, _57.nvm, _57.si, _57_Asserting_0.doc, _57_Asserting_0.tim, _57_Asserting_0.tip, _58.fdt, _58.fdx, _58.fnm, _58.nvd, _58.nvm, _58.si, _58_Asserting_0.doc, _58_Asserting_0.tim, _58_Asserting_0.tip, _59.fdt, _59.fdx, _59.fnm, _59.nvd, _59.nvm, _59.si, _59_Asserting_0.doc, _59_Asserting_0.tim, _59_Asserting_0.tip, _5a.fdt, _5a.fdx, _5a.fnm, _5a.nvd, _5a.nvm, _5a.si, _5a_Asserting_0.doc, _5a_Asserting_0.tim, _5a_Asserting_0.tip, _5b.fdt, _5b.fdx, _5b.fnm, _5b.nvd, _5b.nvm, _5b.si, _5b_Asserting_0.doc, _5b_Asserting_0.tim, _5b_Asserting_0.tip, _5c.fdt, _5c.fdx, _5c.fnm, _5c.nvd, _5c.nvm, _5c.si, _5c_Asserting_0.doc, _5c_Asserting_0.tim, _5c_Asserting_0.tip, _5d.fdt, _5d.fdx, _5d.fnm, _5d.nvd, _5d.nvm, _5d.si, _5d_Asserting_0.doc, _5d_Asserting_0.tim, _5d_Asserting_0.tip, _5e.fdt, _5e.fdx, _5e.fnm, _5e.nvd, _5e.nvm, _5e.si, _5e_Asserting_0.doc, _5e_Asserting_0.tim, _5e_Asserting_0.tip, _5f.fdt, _5f.fdx, _5f.fnm, _5f.nvd, _5f.nvm, _5f.si, _5f_Asserting_0.doc, _5f_Asserting_0.tim, _5f_Asserting_0.tip, _5g.fdt, _5g.fdx, _5g.fnm, _5g.nvd, _5g.nvm, _5g.si, _5g_Asserting_0.doc, _5g_Asserting_0.tim, _5g_Asserting_0.tip, _5h.fdt, _5h.fdx, _5h.fnm, _5h.nvd, _5h.nvm, _5h.si, _5h_Asserting_0.doc, _5h_Asserting_0.tim, _5h_Asserting_0.tip, _5i.fdt, _5i.fdx, _5i.fnm, _5i.nvd, _5i.nvm, _5i.si, _5i_Asserting_0.doc, _5i_Asserting_0.tim, _5i_Asserting_0.tip, _5j.fdt, _5j.fdx, _5j.fnm, _5j.nvd, _5j.nvm, _5j.si, _5j_Asserting_0.doc, _5j_Asserting_0.tim, _5j_Asserting_0.tip, _5k.fdt, _5k.fdx, _5k.fnm, _5k.nvd, _5k.nvm, _5k.si, _5k_Asserting_0.doc, _5k_Asserting_0.tim, _5k_Asserting_0.tip, _5l.fdt, _5l.fdx, _5l.fnm, _5l.nvd, _5l.nvm, _5l.si, _5l_Asserting_0.doc, _5l_Asserting_0.tim, _5l_Asserting_0.tip, _5m.fdt, _5m.fdx, _5m.fnm, _5m.nvd, _5m.nvm, _5m.si, _5m_Asserting_0.doc, _5m_Asserting_0.tim, _5m_Asserting_0.tip, _5n.fdt, _5n.fdx, _5n.fnm, _5n.nvd, _5n.nvm, _5n.si, _5n_Asserting_0.doc, _5n_Asserting_0.tim, _5n_Asserting_0.tip, _5o.fdt, _5o.fdx, _5o.fnm, _5o.nvd, _5o.nvm, _5o.si, _5o_Asserting_0.doc, _5o_Asserting_0.tim, _5o_Asserting_0.tip, _5p.fdt, _5p.fdx, _5p.fnm, _5p.nvd, _5p.nvm, _5p.si, _5p_Asserting_0.doc, _5p_Asserting_0.tim, _5p_Asserting_0.tip, _5q.fdt, _5q.fdx, _5q.fnm, _5q.nvd, _5q.nvm, _5q.si, _5q_Asserting_0.doc, _5q_Asserting_0.tim, _5q_Asserting_0.tip, _5r.fdt, _5r.fdx, _5r.fnm, _5r.nvd, _5r.nvm, _5r.si, _5r_Asserting_0.doc, _5r_Asserting_0.tim, _5r_Asserting_0.tip, _5s.fdt, _5s.fdx, _5s.fnm, _5s.nvd, _5s.nvm, _5s.si, _5s_Asserting_0.doc, _5s_Asserting_0.tim, _5s_Asserting_0.tip, _5t.fdt, _5t.fdx, _5t.fnm, _5t.nvd, _5t.nvm, _5t.si, _5t_Asserting_0.doc, _5t_Asserting_0.tim, _5t_Asserting_0.tip, _5u.fdt, _5u.fdx, _5u.fnm, _5u.nvd, _5u.nvm, _5u.si, _5u_Asserting_0.doc, _5u_Asserting_0.tim, _5u_Asserting_0.tip, _5v.fdt, _5v.fdx, _5v.fnm, _5v.nvd, _5v.nvm, _5v.si, _5v_Asserting_0.doc, _5v_Asserting_0.tim, _5v_Asserting_0.tip, segments_2]}]> but was:<[{indexVersion=1445818330639,generation=2,filelist=[_2d.fdt, _2d.fdx, _2d.fnm, _2d.nvd, _2d.nvm, _2d.si, _2d_Asserting_0.doc, _2d_Asserting_0.tim, _2d_Asserting_0.tip, _4p.fdt, _4p.fdx, _4p.fnm, _4p.nvd, _4p.nvm, _4p.si, _4p_Asserting_0.doc, _4p_Asserting_0.tim, _4p_Asserting_0.tip, _4q.fdt, _4q.fdx, _4q.fnm, _4q.nvd, _4q.nvm, _4q.si, _4q_Asserting_0.doc, _4q_Asserting_0.tim, _4q_Asserting_0.tip, _4r.fdt, _4r.fdx, _4r.fnm, _4r.nvd, _4r.nvm, _4r.si, _4r_Asserting_0.doc, _4r_Asserting_0.tim, _4r_Asserting_0.tip, _4s.fdt, _4s.fdx, _4s.fnm, _4s.nvd, _4s.nvm, _4s.si, _4s_Asserting_0.doc, _4s_Asserting_0.tim, _4s_Asserting_0.tip, _4t.fdt, _4t.fdx, _4t.fnm, _4t.nvd, _4t.nvm, _4t.si, _4t_Asserting_0.doc, _4t_Asserting_0.tim, _4t_Asserting_0.tip, _4u.fdt, _4u.fdx, _4u.fnm, _4u.nvd, _4u.nvm, _4u.si, _4u_Asserting_0.doc, _4u_Asserting_0.tim, _4u_Asserting_0.tip, _4v.fdt, _4v.fdx, _4v.fnm, _4v.nvd, _4v.nvm, _4v.si, _4v_Asserting_0.doc, _4v_Asserting_0.tim, _4v_Asserting_0.tip, _4w.fdt, _4w.fdx, _4w.fnm, _4w.nvd, _4w.nvm, _4w.si, _4w_Asserting_0.doc, _4w_Asserting_0.tim, _4w_Asserting_0.tip, _4x.fdt, _4x.fdx, _4x.fnm, _4x.nvd, _4x.nvm, _4x.si, _4x_Asserting_0.doc, _4x_Asserting_0.tim, _4x_Asserting_0.tip, _4y.fdt, _4y.fdx, _4y.fnm, _4y.nvd, _4y.nvm, _4y.si, _4y_Asserting_0.doc, _4y_Asserting_0.tim, _4y_Asserting_0.tip, _4z.fdt, _4z.fdx, _4z.fnm, _4z.nvd, _4z.nvm, _4z.si, _4z_Asserting_0.doc, _4z_Asserting_0.tim, _4z_Asserting_0.tip, _50.fdt, _50.fdx, _50.fnm, _50.nvd, _50.nvm, _50.si, _50_Asserting_0.doc, _50_Asserting_0.tim, _50_Asserting_0.tip, _51.fdt, _51.fdx, _51.fnm, _51.nvd, _51.nvm, _51.si, _51_Asserting_0.doc, _51_Asserting_0.tim, _51_Asserting_0.tip, _52.fdt, _52.fdx, _52.fnm, _52.nvd, _52.nvm, _52.si, _52_Asserting_0.doc, _52_Asserting_0.tim, _52_Asserting_0.tip, _53.fdt, _53.fdx, _53.fnm, _53.nvd, _53.nvm, _53.si, _53_Asserting_0.doc, _53_Asserting_0.tim, _53_Asserting_0.tip, _54.fdt, _54.fdx, _54.fnm, _54.nvd, _54.nvm, _54.si, _54_Asserting_0.doc, _54_Asserting_0.tim, _54_Asserting_0.tip, _55.fdt, _55.fdx, _55.fnm, _55.nvd, _55.nvm, _55.si, _55_Asserting_0.doc, _55_Asserting_0.tim, _55_Asserting_0.tip, _56.fdt, _56.fdx, _56.fnm, _56.nvd, _56.nvm, _56.si, _56_Asserting_0.doc, _56_Asserting_0.tim, _56_Asserting_0.tip, _57.fdt, _57.fdx, _57.fnm, _57.nvd, _57.nvm, _57.si, _57_Asserting_0.doc, _57_Asserting_0.tim, _57_Asserting_0.tip, _58.fdt, _58.fdx, _58.fnm, _58.nvd, _58.nvm, _58.si, _58_Asserting_0.doc, _58_Asserting_0.tim, _58_Asserting_0.tip, _59.fdt, _59.fdx, _59.fnm, _59.nvd, _59.nvm, _59.si, _59_Asserting_0.doc, _59_Asserting_0.tim, _59_Asserting_0.tip, _5a.fdt, _5a.fdx, _5a.fnm, _5a.nvd, _5a.nvm, _5a.si, _5a_Asserting_0.doc, _5a_Asserting_0.tim, _5a_Asserting_0.tip, _5b.fdt, _5b.fdx, _5b.fnm, _5b.nvd, _5b.nvm, _5b.si, _5b_Asserting_0.doc, _5b_Asserting_0.tim, _5b_Asserting_0.tip, _5c.fdt, _5c.fdx, _5c.fnm, _5c.nvd, _5c.nvm, _5c.si, _5c_Asserting_0.doc, _5c_Asserting_0.tim, _5c_Asserting_0.tip, _5d.fdt, _5d.fdx, _5d.fnm, _5d.nvd, _5d.nvm, _5d.si, _5d_Asserting_0.doc, _5d_Asserting_0.tim, _5d_Asserting_0.tip, _5e.fdt, _5e.fdx, _5e.fnm, _5e.nvd, _5e.nvm, _5e.si, _5e_Asserting_0.doc, _5e_Asserting_0.tim, _5e_Asserting_0.tip, _5f.fdt, _5f.fdx, _5f.fnm, _5f.nvd, _5f.nvm, _5f.si, _5f_Asserting_0.doc, _5f_Asserting_0.tim, _5f_Asserting_0.tip, _5g.fdt, _5g.fdx, _5g.fnm, _5g.nvd, _5g.nvm, _5g.si, _5g_Asserting_0.doc, _5g_Asserting_0.tim, _5g_Asserting_0.tip, _5h.fdt, _5h.fdx, _5h.fnm, _5h.nvd, _5h.nvm, _5h.si, _5h_Asserting_0.doc, _5h_Asserting_0.tim, _5h_Asserting_0.tip, _5i.fdt, _5i.fdx, _5i.fnm, _5i.nvd, _5i.nvm, _5i.si, _5i_Asserting_0.doc, _5i_Asserting_0.tim, _5i_Asserting_0.tip, _5j.fdt, _5j.fdx, _5j.fnm, _5j.nvd, _5j.nvm, _5j.si, _5j_Asserting_0.doc, _5j_Asserting_0.tim, _5j_Asserting_0.tip, _5k.fdt, _5k.fdx, _5k.fnm, _5k.nvd, _5k.nvm, _5k.si, _5k_Asserting_0.doc, _5k_Asserting_0.tim, _5k_Asserting_0.tip, _5l.fdt, _5l.fdx, _5l.fnm, _5l.nvd, _5l.nvm, _5l.si, _5l_Asserting_0.doc, _5l_Asserting_0.tim, _5l_Asserting_0.tip, _5m.fdt, _5m.fdx, _5m.fnm, _5m.nvd, _5m.nvm, _5m.si, _5m_Asserting_0.doc, _5m_Asserting_0.tim, _5m_Asserting_0.tip, _5n.fdt, _5n.fdx, _5n.fnm, _5n.nvd, _5n.nvm, _5n.si, _5n_Asserting_0.doc, _5n_Asserting_0.tim, _5n_Asserting_0.tip, _5o.fdt, _5o.fdx, _5o.fnm, _5o.nvd, _5o.nvm, _5o.si, _5o_Asserting_0.doc, _5o_Asserting_0.tim, _5o_Asserting_0.tip, _5p.fdt, _5p.fdx, _5p.fnm, _5p.nvd, _5p.nvm, _5p.si, _5p_Asserting_0.doc, _5p_Asserting_0.tim, _5p_Asserting_0.tip, _5q.fdt, _5q.fdx, _5q.fnm, _5q.nvd, _5q.nvm, _5q.si, _5q_Asserting_0.doc, _5q_Asserting_0.tim, _5q_Asserting_0.tip, _5r.fdt, _5r.fdx, _5r.fnm, _5r.nvd, _5r.nvm, _5r.si, _5r_Asserting_0.doc, _5r_Asserting_0.tim, _5r_Asserting_0.tip, _5s.fdt, _5s.fdx, _5s.fnm, _5s.nvd, _5s.nvm, _5s.si, _5s_Asserting_0.doc, _5s_Asserting_0.tim, _5s_Asserting_0.tip, _5t.fdt, _5t.fdx, _5t.fnm, _5t.nvd, _5t.nvm, _5t.si, _5t_Asserting_0.doc, _5t_Asserting_0.tim, _5t_Asserting_0.tip, _5u.fdt, _5u.fdx, _5u.fnm, _5u.nvd, _5u.nvm, _5u.si, _5u_Asserting_0.doc, _5u_Asserting_0.tim, _5u_Asserting_0.tip, _5v.fdt, _5v.fdx, _5v.fnm, _5v.nvd, _5v.nvm, _5v.si, _5v_Asserting_0.doc, _5v_Asserting_0.tim, _5v_Asserting_0.tip, segments_2]}, {indexVersion=1445818330639,generation=3,filelist=[_2d.fdt, _2d.fdx, _2d.fnm, _2d.nvd, _2d.nvm, _2d.si, _2d_Asserting_0.doc, _2d_Asserting_0.tim, _2d_Asserting_0.tip, _4q.fdt, _4q.fdx, _4q.fnm, _4q.nvd, _4q.nvm, _4q.si, _4q_Asserting_0.doc, _4q_Asserting_0.tim, _4q_Asserting_0.tip, _5w.cfe, _5w.cfs, _5w.si, segments_3]}]>
	at __randomizedtesting.SeedInfo.seed([A6D967FEF6F05DCB:830E7CCE86B853C8]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.failNotEquals(Assert.java:647)
	at org.junit.Assert.assertEquals(Assert.java:128)
	at org.junit.Assert.assertEquals(Assert.java:147)
	at org.apache.solr.handler.TestReplicationHandler.doTestReplicateAfterCoreReload(TestReplicationHandler.java:1138)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1665)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:864)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:900)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:914)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:873)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:775)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:809)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:820)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR

Error Message:
Captured an uncaught exception in thread: Thread[id=11480, name=coreZkRegister-3481-thread-1, state=RUNNABLE, group=TGRP-LeaderInitiatedRecoveryOnShardRestartTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=11480, name=coreZkRegister-3481-thread-1, state=RUNNABLE, group=TGRP-LeaderInitiatedRecoveryOnShardRestartTest]
Caused by: java.lang.AssertionError
	at __randomizedtesting.SeedInfo.seed([A6D967FEF6F05DCB]:0)
	at org.apache.solr.cloud.ZkController.updateLeaderInitiatedRecoveryState(ZkController.java:2133)
	at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:434)
	at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)
	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)
	at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:346)
	at org.apache.solr.cloud.ZkController.joinElection(ZkController.java:1113)
	at org.apache.solr.cloud.ZkController.register(ZkController.java:926)
	at org.apache.solr.cloud.ZkController.register(ZkController.java:881)
	at org.apache.solr.core.ZkContainer$2.run(ZkContainer.java:183)
	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 10787 lines...]
   [junit4] Suite: org.apache.solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_A6D967FEF6F05DCB-001/init-core-data-001
   [junit4]   2> 1971994 INFO  (SUITE-LeaderInitiatedRecoveryOnShardRestartTest-seed#[A6D967FEF6F05DCB]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /j_/
   [junit4]   2> 1971998 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 1971999 INFO  (Thread-4444) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1971999 INFO  (Thread-4444) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 1972099 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.ZkTestServer start zk server on port:55034
   [junit4]   2> 1972099 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1972099 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1972102 INFO  (zkCallback-1773-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@512a9ce7 name:ZooKeeperConnection Watcher:127.0.0.1:55034 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1972102 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1972103 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1972103 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 1972105 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1972106 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1972107 INFO  (zkCallback-1774-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6f80947e name:ZooKeeperConnection Watcher:127.0.0.1:55034/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1972107 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1972108 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1972108 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 1972109 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 1972111 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 1972112 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 1972114 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 1972114 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 1972117 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 1972117 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 1972119 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1972120 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1972122 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 1972122 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 1972124 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 1972124 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 1972126 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 1972126 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 1972129 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 1972130 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 1972131 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1972132 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1972139 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1972139 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1972141 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 1972141 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 1972142 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 1972143 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 1972431 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_A6D967FEF6F05DCB-001/control-001/cores/collection1
   [junit4]   2> 1972433 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 1972435 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@7913eec{/j_,null,AVAILABLE}
   [junit4]   2> 1972435 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.e.j.s.ServerConnector Started ServerConnector@6244865a{HTTP/1.1}{127.0.0.1:47317}
   [junit4]   2> 1972435 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.e.j.s.Server Started @1975402ms
   [junit4]   2> 1972435 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/j_, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_A6D967FEF6F05DCB-001/tempDir-001/control/data, hostPort=47317, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_A6D967FEF6F05DCB-001/control-001/cores}
   [junit4]   2> 1972436 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@6b482747
   [junit4]   2> 1972436 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_A6D967FEF6F05DCB-001/control-001/'
   [junit4]   2> 1972436 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1972436 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1972453 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1972453 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1972455 INFO  (zkCallback-1775-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6c83ceb name:ZooKeeperConnection Watcher:127.0.0.1:55034/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1972455 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1972455 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1972456 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1972456 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_A6D967FEF6F05DCB-001/control-001/solr.xml
   [junit4]   2> 1972470 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_A6D967FEF6F05DCB-001/control-001/cores
   [junit4]   2> 1972470 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.CoreContainer New CoreContainer 1243423389
   [junit4]   2> 1972470 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_A6D967FEF6F05DCB-001/control-001/]
   [junit4]   2> 1972470 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_A6D967FEF6F05DCB-001/control-001/lib
   [junit4]   2> 1972470 WARN  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_A6D967FEF6F05DCB-001/control-001/lib).
   [junit4]   2> 1972478 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1972479 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1972480 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1972480 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1972480 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:55034/solr
   [junit4]   2> 1972480 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 1972480 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1972480 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1972488 INFO  (zkCallback-1777-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@f1b0752 name:ZooKeeperConnection Watcher:127.0.0.1:55034 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1972488 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1972489 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1972491 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [n:127.0.0.1:47317_j_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1972492 INFO  (zkCallback-1778-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@543d4d5f name:ZooKeeperConnection Watcher:127.0.0.1:55034/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1972492 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [n:127.0.0.1:47317_j_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1972493 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [n:127.0.0.1:47317_j_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 1972495 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [n:127.0.0.1:47317_j_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 1972500 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [n:127.0.0.1:47317_j_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 1972502 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [n:127.0.0.1:47317_j_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 1972504 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [n:127.0.0.1:47317_j_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 1972506 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [n:127.0.0.1:47317_j_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 1972508 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [n:127.0.0.1:47317_j_    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 1972509 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [n:127.0.0.1:47317_j_    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 1972510 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [n:127.0.0.1:47317_j_    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 1972511 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [n:127.0.0.1:47317_j_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:47317_j_
   [junit4]   2> 1972511 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [n:127.0.0.1:47317_j_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:47317_j_
   [junit4]   2> 1972513 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [n:127.0.0.1:47317_j_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 1972514 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [n:127.0.0.1:47317_j_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 1972515 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [n:127.0.0.1:47317_j_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1972516 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [n:127.0.0.1:47317_j_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94753102892564484-127.0.0.1:47317_j_-n_0000000000
   [junit4]   2> 1972516 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [n:127.0.0.1:47317_j_    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:47317_j_
   [junit4]   2> 1972516 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [n:127.0.0.1:47317_j_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 1972518 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [n:127.0.0.1:47317_j_    ] o.a.s.c.Overseer Overseer (id=94753102892564484-127.0.0.1:47317_j_-n_0000000000) starting
   [junit4]   2> 1972519 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [n:127.0.0.1:47317_j_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 1972526 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [n:127.0.0.1:47317_j_    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 1972526 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [n:127.0.0.1:47317_j_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1972526 INFO  (OverseerCollectionConfigSetProcessor-94753102892564484-127.0.0.1:47317_j_-n_0000000000) [n:127.0.0.1:47317_j_    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 1972526 INFO  (OverseerStateUpdate-94753102892564484-127.0.0.1:47317_j_-n_0000000000) [n:127.0.0.1:47317_j_    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 1972547 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [n:127.0.0.1:47317_j_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1972547 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [n:127.0.0.1:47317_j_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1972548 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [n:127.0.0.1:47317_j_    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_A6D967FEF6F05DCB-001/control-001/cores
   [junit4]   2> 1972549 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [n:127.0.0.1:47317_j_    ] o.a.s.c.SolrCore Created CoreDescriptor: {schema=schema.xml, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_A6D967FEF6F05DCB-001/control-001/cores/collection1, transient=false, shard=, name=collection1, config=solrconfig.xml, loadOnStartup=true, collection=control_collection, dataDir=data/, coreNodeName=, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_A6D967FEF6F05DCB-001/control-001/cores/collection1/, configSetProperties=configsetprops.json}
   [junit4]   2> 1972549 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [n:127.0.0.1:47317_j_    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_A6D967FEF6F05DCB-001/control-001/cores/collection1/
   [junit4]   2> 1972549 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [n:127.0.0.1:47317_j_    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 1972551 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 1972552 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1972552 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [n:127.0.0.1:47317_j_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 1972552 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [n:127.0.0.1:47317_j_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1972554 INFO  (OverseerStateUpdate-94753102892564484-127.0.0.1:47317_j_-n_0000000000) [n:127.0.0.1:47317_j_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:47317/j_",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "node_name":"127.0.0.1:47317_j_",
   [junit4]   2>   "state":"down"} current state version: 0
   [junit4]   2> 1972554 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 1972554 INFO  (OverseerStateUpdate-94753102892564484-127.0.0.1:47317_j_-n_0000000000) [n:127.0.0.1:47317_j_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:47317/j_",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "node_name":"127.0.0.1:47317_j_",
   [junit4]   2>   "state":"down"}
   [junit4]   2> 1972555 INFO  (OverseerStateUpdate-94753102892564484-127.0.0.1:47317_j_-n_0000000000) [n:127.0.0.1:47317_j_    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 1972555 INFO  (OverseerStateUpdate-94753102892564484-127.0.0.1:47317_j_-n_0000000000) [n:127.0.0.1:47317_j_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 1972558 INFO  (zkCallback-1778-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1973555 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1973555 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 1973557 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 1973557 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 1973557 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1973557 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_A6D967FEF6F05DCB-001/control-001/cores/collection1/'
   [junit4]   2> 1973558 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1973558 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1973595 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 1973603 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 1973617 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 1973677 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1973678 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 1973685 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1974253 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 1974262 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 1974267 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 1974281 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1974284 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1974294 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1974295 WARN  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 1974295 WARN  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 1974295 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1974297 WARN  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 1974298 WARN  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 1974299 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 1974299 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 1974299 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 1974299 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_A6D967FEF6F05DCB-001/control-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 1974299 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@2a147b53
   [junit4]   2> 1974300 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_A6D967FEF6F05DCB-001/control-001/cores/collection1/data
   [junit4]   2> 1974308 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_A6D967FEF6F05DCB-001/control-001/cores/collection1/data/index/
   [junit4]   2> 1974308 WARN  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_A6D967FEF6F05DCB-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 1974308 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_A6D967FEF6F05DCB-001/control-001/cores/collection1/data/index
   [junit4]   2> 1974309 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=18, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=1.2626953125, noCFSRatio=1.0]
   [junit4]   2> 1974309 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@6e942479 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@3fe6bca6),segFN=segments_1,generation=1}
   [junit4]   2> 1974309 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 1974324 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 1974324 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 1974324 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 1974324 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1974324 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1974324 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 1974325 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1974325 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1974325 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 1974327 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_A6D967FEF6F05DCB-001/control-001/cores/collection1/data/
   [junit4]   2> 1974331 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_A6D967FEF6F05DCB-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 1974332 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1974334 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1974335 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1974336 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1974359 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /update/json,/admin/threads,/replication,/config,/admin/mbeans,/admin/properties,/get,standard,/admin/logging,/admin/ping,/admin/file,/update/json/docs,/admin/segments,/schema,/admin/system,/update/csv,/update,/admin/plugins,/admin/luke
   [junit4]   2> 1974367 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 1974388 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1974388 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1974389 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 1974389 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 1974389 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@1184a6ad
   [junit4]   2> 1974399 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@6e942479 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@3fe6bca6),segFN=segments_1,generation=1}
   [junit4]   2> 1974399 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 1974399 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@29ed5726[collection1] main
   [junit4]   2> 1974400 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 1974423 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1974423 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 1974424 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 1974424 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 1974424 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1974425 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 1974425 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1974425 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 1974425 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1974425 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 1974426 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 1974426 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 1974426 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1516049648824877056
   [junit4]   2> 1974432 INFO  (searcherExecutor-3444-thread-1-processing-x:collection1 r:core_node1 s:shard1 c:control_collection n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@29ed5726[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1974433 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 7.0ms to seed version buckets with highest version 1516049648824877056
   [junit4]   2> 1974433 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 1974440 INFO  (coreLoadExecutor-3443-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 1974440 INFO  (coreZkRegister-3437-thread-1-processing-x:collection1 r:core_node1 s:shard1 c:control_collection n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:47317/j_ collection:control_collection shard:shard1
   [junit4]   2> 1974442 INFO  (coreZkRegister-3437-thread-1-processing-x:collection1 r:core_node1 s:shard1 c:control_collection n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 1974446 INFO  (coreZkRegister-3437-thread-1-processing-x:collection1 r:core_node1 s:shard1 c:control_collection n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94753102892564484-core_node1-n_0000000000
   [junit4]   2> 1974447 INFO  (coreZkRegister-3437-thread-1-processing-x:collection1 r:core_node1 s:shard1 c:control_collection n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 1974449 INFO  (coreZkRegister-3437-thread-1-processing-x:collection1 r:core_node1 s:shard1 c:control_collection n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1974449 INFO  (coreZkRegister-3437-thread-1-processing-x:collection1 r:core_node1 s:shard1 c:control_collection n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1974449 INFO  (coreZkRegister-3437-thread-1-processing-x:collection1 r:core_node1 s:shard1 c:control_collection n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:47317/j_/collection1/
   [junit4]   2> 1974449 INFO  (OverseerStateUpdate-94753102892564484-127.0.0.1:47317_j_-n_0000000000) [n:127.0.0.1:47317_j_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 1974449 INFO  (coreZkRegister-3437-thread-1-processing-x:collection1 r:core_node1 s:shard1 c:control_collection n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 1974449 INFO  (coreZkRegister-3437-thread-1-processing-x:collection1 r:core_node1 s:shard1 c:control_collection n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:47317/j_/collection1/ has no replicas
   [junit4]   2> 1974449 INFO  (coreZkRegister-3437-thread-1-processing-x:collection1 r:core_node1 s:shard1 c:control_collection n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:47317/j_/collection1/ shard1
   [junit4]   2> 1974450 INFO  (zkCallback-1778-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1974453 INFO  (coreZkRegister-3437-thread-1-processing-x:collection1 r:core_node1 s:shard1 c:control_collection n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 1974456 INFO  (coreZkRegister-3437-thread-1-processing-x:collection1 r:core_node1 s:shard1 c:control_collection n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 1974458 INFO  (OverseerStateUpdate-94753102892564484-127.0.0.1:47317_j_-n_0000000000) [n:127.0.0.1:47317_j_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:47317/j_",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 2
   [junit4]   2> 1974561 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1974562 INFO  (zkCallback-1778-thread-1-processing-n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1974566 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1974568 INFO  (zkCallback-1780-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@77d5e3e1 name:ZooKeeperConnection Watcher:127.0.0.1:55034/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1974568 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1974569 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1974569 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1974577 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 1974612 INFO  (coreZkRegister-3437-thread-1-processing-x:collection1 r:core_node1 s:shard1 c:control_collection n:127.0.0.1:47317_j_) [n:127.0.0.1:47317_j_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:47317/j_/collection1/ and leader is http://127.0.0.1:47317/j_/collection1/
   [junit4]   2> 1974612 INFO  (coreZkRegister-3437-thread-1-processing-x:collection1 r:core_node1 s:shard1 c:control_collection n:127.0.0.1:4731

[...truncated too long message...]

ndex
   [junit4]   2> 2700555 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[A6D967FEF6F05DCB]) [    x:collection1] o.a.s.c.CachingDirectoryFactory looking to close /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A6D967FEF6F05DCB-001/solr-instance-027/collection1/data [CachedDir<<refCount=0;path=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A6D967FEF6F05DCB-001/solr-instance-027/collection1/data;done=false>>]
   [junit4]   2> 2700555 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[A6D967FEF6F05DCB]) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A6D967FEF6F05DCB-001/solr-instance-027/collection1/data
   [junit4]   2> 2700555 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[A6D967FEF6F05DCB]) [    x:collection1] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@31b1f6e6{/solr,null,UNAVAILABLE}
   [junit4]   2> 2700562 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[A6D967FEF6F05DCB]) [    ] o.e.j.s.ServerConnector Stopped ServerConnector@f0c3f60{HTTP/1.1}{127.0.0.1:0}
   [junit4]   2> 2700563 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[A6D967FEF6F05DCB]) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1239272902
   [junit4]   2> 2700563 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[A6D967FEF6F05DCB]) [    x:collection1] o.a.s.c.SolrCore [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@9e5cb00
   [junit4]   2> 2700563 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[A6D967FEF6F05DCB]) [    x:collection1] o.a.s.u.UpdateHandler closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
   [junit4]   2> 2700563 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[A6D967FEF6F05DCB]) [    x:collection1] o.a.s.u.SolrCoreState Closing SolrCoreState
   [junit4]   2> 2700563 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[A6D967FEF6F05DCB]) [    x:collection1] o.a.s.u.DefaultSolrCoreState SolrCoreState ref count has reached 0 - closing IndexWriter
   [junit4]   2> 2700563 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[A6D967FEF6F05DCB]) [    x:collection1] o.a.s.u.DefaultSolrCoreState closing IndexWriter with IndexWriterCloser
   [junit4]   2> 2700566 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[A6D967FEF6F05DCB]) [    x:collection1] o.a.s.c.SolrCore [collection1] Closing main searcher on request.
   [junit4]   2> 2700566 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[A6D967FEF6F05DCB]) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing StandardDirectoryFactory - 2 directories currently being tracked
   [junit4]   2> 2700566 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[A6D967FEF6F05DCB]) [    x:collection1] o.a.s.c.CachingDirectoryFactory looking to close /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A6D967FEF6F05DCB-001/solr-instance-028/collection1/data [CachedDir<<refCount=0;path=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A6D967FEF6F05DCB-001/solr-instance-028/collection1/data;done=false>>]
   [junit4]   2> 2700566 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[A6D967FEF6F05DCB]) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A6D967FEF6F05DCB-001/solr-instance-028/collection1/data
   [junit4]   2> 2700567 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[A6D967FEF6F05DCB]) [    x:collection1] o.a.s.c.CachingDirectoryFactory looking to close /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A6D967FEF6F05DCB-001/solr-instance-028/collection1/data/index [CachedDir<<refCount=0;path=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A6D967FEF6F05DCB-001/solr-instance-028/collection1/data/index;done=false>>]
   [junit4]   2> 2700567 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[A6D967FEF6F05DCB]) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A6D967FEF6F05DCB-001/solr-instance-028/collection1/data/index
   [junit4]   2> 2700567 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[A6D967FEF6F05DCB]) [    x:collection1] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@927b9a2{/solr,null,UNAVAILABLE}
   [junit4]   2> 2700569 INFO  (SUITE-TestReplicationHandler-seed#[A6D967FEF6F05DCB]-worker) [    ] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A6D967FEF6F05DCB-001
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene53), sim=RandomSimilarityProvider(queryNorm=true,coord=yes): {}, locale=en, timezone=America/Rankin_Inlet
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_72 (64-bit)/cpus=4,threads=1,free=233224352,total=457703424
   [junit4]   2> NOTE: All tests run in this JVM: [ReplicationFactorTest, IndexSchemaTest, TestPhraseSuggestions, BigEndianAscendingWordDeserializerTest, TestValueSourceCache, CleanupOldIndexTest, HdfsDirectoryFactoryTest, OverriddenZkACLAndCredentialsProvidersTest, TestDistributedStatsComponentCardinality, SynonymTokenizerTest, CollectionReloadTest, TestSolrJ, TestBinaryField, TestFoldingMultitermQuery, EchoParamsTest, TestDistributedSearch, ZkStateReaderTest, TestConfigSetsAPIExclusivity, TestConfigReload, TestOmitPositions, LeaderInitiatedRecoveryOnCommitTest, QueryElevationComponentTest, TestHighFrequencyDictionaryFactory, SignatureUpdateProcessorFactoryTest, PathHierarchyTokenizerFactoryTest, TestRemoteStreaming, TestTolerantSearch, TestIBSimilarityFactory, TestCollectionAPI, JavabinLoaderTest, TestBadConfig, TestRandomFaceting, TestDocumentBuilder, TestSearcherReuse, SolrCoreTest, TestJsonFacets, DistributedFacetPivotSmallAdvancedTest, TestLMDirichletSimilarityFactory, OverseerCollectionConfigSetProcessorTest, TestDistributedMissingSort, ResponseHeaderTest, HdfsBasicDistributedZk2Test, TestExclusionRuleCollectionAccess, HdfsUnloadDistributedZkTest, BasicAuthIntegrationTest, ReturnFieldsTest, TestFieldSortValues, PingRequestHandlerTest, DataDrivenBlockJoinTest, TestCollapseQParserPlugin, CloudExitableDirectoryReaderTest, AnalysisAfterCoreReloadTest, SolrRequestParserTest, TestCollationFieldDocValues, HighlighterMaxOffsetTest, TestMiniSolrCloudClusterBase, TestRandomDVFaceting, MigrateRouteKeyTest, TestRawTransformer, CacheHeaderTest, DisMaxRequestHandlerTest, SSLMigrationTest, ResourceLoaderTest, PrimUtilsTest, TestDocBasedVersionConstraints, TestManagedSynonymFilterFactory, FileUtilsTest, SyncSliceTest, SpellingQueryConverterTest, TestJoin, FullHLLTest, TestCustomSort, TestInitParams, DistributedQueryElevationComponentTest, TestConfig, TestAtomicUpdateErrorCases, OutputWriterTest, JsonLoaderTest, LeaderFailoverAfterPartitionTest, BasicDistributedZkTest, TestCloudSchemaless, StandardRequestHandlerTest, BadCopyFieldTest, TestSchemaSimilarityResource, IndexSchemaRuntimeFieldTest, TestRangeQuery, SolrCloudExampleTest, SolrTestCaseJ4Test, TestReplicationHandler]
   [junit4] Completed [396/543] on J0 in 660.67s, 13 tests, 1 failure <<< FAILURES!

[...truncated 492 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 543 suites (7 ignored), 2144 tests, 1 error, 1 failure, 31 ignored (18 assumptions) [seed: A6D967FEF6F05DCB]

Total time: 201 minutes 18 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 996 - Still Failing

[Apache Jenkins Server <je...@builds.apache.org>]

Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/996/

3 tests failed.
FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Error from server at http://127.0.0.1:38621: Could not load collection from ZK:acollectionafterbaddelete

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:38621: Could not load collection from ZK:acollectionafterbaddelete
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.makeRequest(CollectionsAPIDistributedZkTest.java:302)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testErrorHandling(CollectionsAPIDistributedZkTest.java:419)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test(CollectionsAPIDistributedZkTest.java:169)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1665)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:864)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:900)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:914)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:963)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:938)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:873)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:775)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:809)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:820)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR

Error Message:
Captured an uncaught exception in thread: Thread[id=20121, name=coreZkRegister-6242-thread-1, state=RUNNABLE, group=TGRP-LeaderInitiatedRecoveryOnShardRestartTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=20121, name=coreZkRegister-6242-thread-1, state=RUNNABLE, group=TGRP-LeaderInitiatedRecoveryOnShardRestartTest]
Caused by: java.lang.AssertionError
	at __randomizedtesting.SeedInfo.seed([6D2393130133B000]:0)
	at org.apache.solr.cloud.ZkController.updateLeaderInitiatedRecoveryState(ZkController.java:2133)
	at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:434)
	at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)
	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)
	at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:346)
	at org.apache.solr.cloud.ZkController.joinElection(ZkController.java:1113)
	at org.apache.solr.cloud.ZkController.register(ZkController.java:926)
	at org.apache.solr.cloud.ZkController.register(ZkController.java:881)
	at org.apache.solr.core.ZkContainer$2.run(ZkContainer.java:183)
	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.cloud.OverseerTest.testExternalClusterStateChangeBehavior

Error Message:
Illegal state, was: down expected:active clusterState:live nodes:[]collections:{c1=DocCollection(c1)={   "router":{"name":"implicit"},   "shards":{"shard1":{       "parent":null,       "state":"active",       "range":null,       "replicas":{"core_node1":{           "base_url":"http://127.0.0.1/solr",           "node_name":"node1",           "core":"core1",           "roles":"",           "state":"down"}}}}}, test=LazyCollectionRef(test)}

Stack Trace:
java.lang.AssertionError: Illegal state, was: down expected:active clusterState:live nodes:[]collections:{c1=DocCollection(c1)={
  "router":{"name":"implicit"},
  "shards":{"shard1":{
      "parent":null,
      "state":"active",
      "range":null,
      "replicas":{"core_node1":{
          "base_url":"http://127.0.0.1/solr",
          "node_name":"node1",
          "core":"core1",
          "roles":"",
          "state":"down"}}}}}, test=LazyCollectionRef(test)}
	at __randomizedtesting.SeedInfo.seed([6D2393130133B000:53D90FFE3A3EA4E]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.solr.cloud.OverseerTest.verifyStatus(OverseerTest.java:601)
	at org.apache.solr.cloud.OverseerTest.testExternalClusterStateChangeBehavior(OverseerTest.java:1261)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1665)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:864)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:900)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:914)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:873)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:775)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:809)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:820)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 10678 lines...]
   [junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_6D2393130133B000-001/init-core-data-001
   [junit4]   2> 1285696 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[6D2393130133B000]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true)
   [junit4]   2> 1285696 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[6D2393130133B000]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   2> 1285701 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 1285701 INFO  (Thread-4477) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1285701 INFO  (Thread-4477) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 1285801 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.ZkTestServer start zk server on port:44976
   [junit4]   2> 1285801 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1285802 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1285804 INFO  (zkCallback-1000-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@8910ea2 name:ZooKeeperConnection Watcher:127.0.0.1:44976 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1285804 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1285805 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1285805 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 1285807 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1285807 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1285808 INFO  (zkCallback-1001-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5c375294 name:ZooKeeperConnection Watcher:127.0.0.1:44976/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1285808 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1285809 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1285809 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 1285810 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 1285812 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 1285813 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 1285815 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 1285815 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 1285817 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 1285817 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 1285819 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1285820 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1285821 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 1285821 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 1285823 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 1285823 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 1285825 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 1285825 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 1285827 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 1285827 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 1285829 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1285829 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1285831 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1285831 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1285832 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 1285833 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 1285834 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 1285834 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 1285836 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1285837 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1285838 INFO  (zkCallback-1002-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@66746ee1 name:ZooKeeperConnection Watcher:127.0.0.1:44976 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1285838 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1285838 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1285838 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 1285840 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1285840 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1285842 INFO  (zkCallback-1003-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7e58604f name:ZooKeeperConnection Watcher:127.0.0.1:44976/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1285842 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1285842 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1285843 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf2/solrconfig.xml
   [junit4]   2> 1285843 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/solrconfig.xml
   [junit4]   2> 1285845 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf2/schema.xml
   [junit4]   2> 1285845 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/schema.xml
   [junit4]   2> 1285847 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf2/enumsConfig.xml
   [junit4]   2> 1285847 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/enumsConfig.xml
   [junit4]   2> 1285849 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf2/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1285849 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1285851 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf2/stopwords.txt
   [junit4]   2> 1285851 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/stopwords.txt
   [junit4]   2> 1285853 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf2/protwords.txt
   [junit4]   2> 1285853 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/protwords.txt
   [junit4]   2> 1285855 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf2/currency.xml
   [junit4]   2> 1285855 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/currency.xml
   [junit4]   2> 1285857 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf2/open-exchange-rates.json
   [junit4]   2> 1285857 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/open-exchange-rates.json
   [junit4]   2> 1285859 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf2/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1285859 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1285861 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf2/old_synonyms.txt
   [junit4]   2> 1285861 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/old_synonyms.txt
   [junit4]   2> 1285863 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf2/synonyms.txt
   [junit4]   2> 1285863 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/synonyms.txt
   [junit4]   2> 1285865 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/elevate.xml to /configs/conf2/elevate.xml
   [junit4]   2> 1285865 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/elevate.xml
   [junit4]   2> 1286156 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_6D2393130133B000-001/control-001/cores/collection1
   [junit4]   2> 1286157 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 1286159 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@73b4d45a{/,null,AVAILABLE}
   [junit4]   2> 1286161 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.e.j.s.ServerConnector Started ServerConnector@17cc8ee0{HTTP/1.1}{127.0.0.1:56839}
   [junit4]   2> 1286161 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.e.j.s.Server Started @1289063ms
   [junit4]   2> 1286161 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_6D2393130133B000-001/control-001/cores, hostPort=56839, hostContext=/, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_6D2393130133B000-001/tempDir-001/control/data}
   [junit4]   2> 1286161 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@6b482747
   [junit4]   2> 1286161 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_6D2393130133B000-001/control-001/'
   [junit4]   2> 1286161 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1286161 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1286177 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1286178 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1286179 INFO  (zkCallback-1004-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2c0fecc4 name:ZooKeeperConnection Watcher:127.0.0.1:44976/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1286179 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1286180 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1286181 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1286181 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_6D2393130133B000-001/control-001/solr.xml
   [junit4]   2> 1286209 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_6D2393130133B000-001/control-001/cores
   [junit4]   2> 1286209 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.CoreContainer New CoreContainer 120356151
   [junit4]   2> 1286209 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_6D2393130133B000-001/control-001/]
   [junit4]   2> 1286209 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_6D2393130133B000-001/control-001/lib
   [junit4]   2> 1286210 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_6D2393130133B000-001/control-001/lib).
   [junit4]   2> 1286217 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1286219 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1286219 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1286219 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1286219 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:44976/solr
   [junit4]   2> 1286219 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 1286219 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1286220 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1286221 INFO  (zkCallback-1006-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@c984492 name:ZooKeeperConnection Watcher:127.0.0.1:44976 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1286222 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1286222 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1286223 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [n:127.0.0.1:56839_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1286224 INFO  (zkCallback-1007-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7f990515 name:ZooKeeperConnection Watcher:127.0.0.1:44976/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1286224 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [n:127.0.0.1:56839_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1286226 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [n:127.0.0.1:56839_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 1286228 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [n:127.0.0.1:56839_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 1286232 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [n:127.0.0.1:56839_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 1286234 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [n:127.0.0.1:56839_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 1286236 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [n:127.0.0.1:56839_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 1286238 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [n:127.0.0.1:56839_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 1286240 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [n:127.0.0.1:56839_    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 1286241 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [n:127.0.0.1:56839_    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 1286243 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [n:127.0.0.1:56839_    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 1286244 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [n:127.0.0.1:56839_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:56839_
   [junit4]   2> 1286244 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [n:127.0.0.1:56839_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:56839_
   [junit4]   2> 1286246 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [n:127.0.0.1:56839_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 1286248 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [n:127.0.0.1:56839_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 1286249 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [n:127.0.0.1:56839_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1286250 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [n:127.0.0.1:56839_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94747307604443142-127.0.0.1:56839_-n_0000000000
   [junit4]   2> 1286250 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [n:127.0.0.1:56839_    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:56839_
   [junit4]   2> 1286250 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [n:127.0.0.1:56839_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 1286251 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [n:127.0.0.1:56839_    ] o.a.s.c.Overseer Overseer (id=94747307604443142-127.0.0.1:56839_-n_0000000000) starting
   [junit4]   2> 1286253 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [n:127.0.0.1:56839_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 1286259 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [n:127.0.0.1:56839_    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 1286260 INFO  (OverseerCollectionConfigSetProcessor-94747307604443142-127.0.0.1:56839_-n_0000000000) [n:127.0.0.1:56839_    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 1286260 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [n:127.0.0.1:56839_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1286260 INFO  (OverseerStateUpdate-94747307604443142-127.0.0.1:56839_-n_0000000000) [n:127.0.0.1:56839_    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 1286281 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [n:127.0.0.1:56839_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1286281 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [n:127.0.0.1:56839_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1286282 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [n:127.0.0.1:56839_    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_6D2393130133B000-001/control-001/cores
   [junit4]   2> 1286283 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [n:127.0.0.1:56839_    ] o.a.s.c.SolrCore Created CoreDescriptor: {absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_6D2393130133B000-001/control-001/cores/collection1/, configSetProperties=configsetprops.json, transient=false, loadOnStartup=true, config=solrconfig.xml, collection=control_collection, coreNodeName=, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_6D2393130133B000-001/control-001/cores/collection1, schema=schema.xml, shard=, name=collection1, dataDir=data/}
   [junit4]   2> 1286283 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [n:127.0.0.1:56839_    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_6D2393130133B000-001/control-001/cores/collection1/
   [junit4]   2> 1286283 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [n:127.0.0.1:56839_    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 1286284 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [n:127.0.0.1:56839_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 1286284 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [n:127.0.0.1:56839_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1286284 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 1286284 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1286286 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 1286286 INFO  (OverseerStateUpdate-94747307604443142-127.0.0.1:56839_-n_0000000000) [n:127.0.0.1:56839_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "base_url":"http://127.0.0.1:56839",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "node_name":"127.0.0.1:56839_",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 1286287 INFO  (OverseerStateUpdate-94747307604443142-127.0.0.1:56839_-n_0000000000) [n:127.0.0.1:56839_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "base_url":"http://127.0.0.1:56839",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "node_name":"127.0.0.1:56839_",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 1286287 INFO  (OverseerStateUpdate-94747307604443142-127.0.0.1:56839_-n_0000000000) [n:127.0.0.1:56839_    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 1286288 INFO  (OverseerStateUpdate-94747307604443142-127.0.0.1:56839_-n_0000000000) [n:127.0.0.1:56839_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 1286289 INFO  (zkCallback-1007-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1287287 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1287287 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 1287287 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 1287287 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 1287288 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1287288 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_6D2393130133B000-001/control-001/cores/collection1/'
   [junit4]   2> 1287288 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1287288 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1287302 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 1287307 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 1287320 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 1287348 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1287349 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 1287355 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1288354 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 1288362 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 1288364 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 1288373 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1288376 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1288379 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1288380 WARN  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 1288380 WARN  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 1288380 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1288380 WARN  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 1288380 WARN  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 1288381 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
   [junit4]   2> 1288382 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 1288382 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 1288382 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_6D2393130133B000-001/control-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 1288382 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@238e338
   [junit4]   2> 1288382 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_6D2393130133B000-001/control-001/cores/collection1/data
   [junit4]   2> 1288383 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_6D2393130133B000-001/control-001/cores/collection1/data/index/
   [junit4]   2> 1288383 WARN  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_6D2393130133B000-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 1288383 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_6D2393130133B000-001/control-001/cores/collection1/data/index
   [junit4]   2> 1288383 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=48, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.5178545787741137]
   [junit4]   2> 1288384 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@6d7e0d74 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@6ea572d4),segFN=segments_1,generation=1}
   [junit4]   2> 1288384 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 1288384 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_6D2393130133B000-001/control-001/cores/collection1/data/
   [junit4]   2> 1288384 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_6D2393130133B000-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 1288389 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 1288389 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 1288389 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 1288389 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1288389 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1288389 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 1288389 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1288390 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1288390 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 1288392 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1288394 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1288395 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1288396 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1288399 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/segments,/update,/schema,/get,/update/json/docs,/admin/properties,/admin/logging,/admin/plugins,/admin/mbeans,/admin/ping,/replication,/config,/admin/system,/update/csv,/admin/luke,/update/json,/admin/threads,/admin/file,standard
   [junit4]   2> 1288401 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 1288401 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1288401 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1288402 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 1288402 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 1288402 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=42, maxMergeAtOnceExplicit=25, maxMergedSegmentMB=62.6435546875, floorSegmentMB=0.212890625, forceMergeDeletesPctAllowed=27.910711213771886, segmentsPerTier=25.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 1288403 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@6d7e0d74 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@6ea572d4),segFN=segments_1,generation=1}
   [junit4]   2> 1288403 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 1288403 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@59f3a728[collection1] main
   [junit4]   2> 1288403 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 1288404 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1288404 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 1288404 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 1288404 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 1288404 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1288405 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 1288405 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1288405 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 1288405 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1288406 INFO  (searcherExecutor-2769-thread-1-processing-s:shard1 r:core_node1 n:127.0.0.1:56839_ x:collection1 c:control_collection) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@59f3a728[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1288406 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 1288406 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 1288406 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 1288406 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1515956924506439680
   [junit4]   2> 1288408 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 2.0ms to seed version buckets with highest version 1515956924506439680
   [junit4]   2> 1288408 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 1288409 INFO  (coreLoadExecutor-2768-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 1288409 INFO  (coreZkRegister-2762-thread-1-processing-s:shard1 r:core_node1 n:127.0.0.1:56839_ x:collection1 c:control_collection) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:56839 collection:control_collection shard:shard1
   [junit4]   2> 1288410 INFO  (coreZkRegister-2762-thread-1-processing-s:shard1 r:core_node1 n:127.0.0.1:56839_ x:collection1 c:control_collection) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 1288414 INFO  (coreZkRegister-2762-thread-1-processing-s:shard1 r:core_node1 n:127.0.0.1:56839_ x:collection1 c:control_collection) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94747307604443142-core_node1-n_0000000000
   [junit4]   2> 1288414 INFO  (coreZkRegister-2762-thread-1-processing-s:shard1 r:core_node1 n:127.0.0.1:56839_ x:collection1 c:control_collection) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 1288417 INFO  (coreZkRegister-2762-thread-1-processing-s:shard1 r:core_node1 n:127.0.0.1:56839_ x:collection1 c:control_collection) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1288417 INFO  (coreZkRegister-2762-thread-1-processing-s:shard1 r:core_node1 n:127.0.0.1:56839_ x:collection1 c:control_collection) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1288417 INFO  (coreZkRegister-2762-thread-1-processing-s:shard1 r:core_node1 n:127.0.0.1:56839_ x:collection1 c:control_collection) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:56839/collection1/
   [junit4]   2> 1288417 INFO  (coreZkRegister-2762-thread-1-processing-s:shard1 r:core_node1 n:127.0.0.1:56839_ x:collection1 c:control_collection) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 1288417 INFO  (OverseerStateUpdate-94747307604443142-127.0.0.1:56839_-n_0000000000) [n:127.0.0.1:56839_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 1288417 INFO  (coreZkRegister-2762-thread-1-processing-s:shard1 r:core_node1 n:127.0.0.1:56839_ x:collection1 c:control_collection) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:56839/collection1/ has no replicas
   [junit4]   2> 1288417 INFO  (coreZkRegister-2762-thread-1-processing-s:shard1 r:core_node1 n:127.0.0.1:56839_ x:collection1 c:control_collection) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:56839/collection1/ shard1
   [junit4]   2> 1288418 INFO  (coreZkRegister-2762-thread-1-processing-s:shard1 r:core_node1 n:127.0.0.1:56839_ x:collection1 c:control_collection) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 1288418 INFO  (zkCallback-1007-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1288420 INFO  (coreZkRegister-2762-thread-1-processing-s:shard1 r:core_node1 n:127.0.0.1:56839_ x:collection1 c:control_collection) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 1288423 INFO  (OverseerStateUpdate-94747307604443142-127.0.0.1:56839_-n_0000000000) [n:127.0.0.1:56839_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:56839",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 2
   [junit4]   2> 1288525 INFO  (zkCallback-1007-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1288574 INFO  (coreZkRegister-2762-thread-1-processing-s:shard1 r:core_node1 n:127.0.0.1:56839_ x:collection1 c:control_collection) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:56839/collection1/ and leader is http://127.0.0.1:56839/collection1/
   [junit4]   2> 1288574 INFO  (coreZkRegister-2762-thread-1-processing-s:shard1 r:core_node1 n:127.0.0.1:56839_ x:collection1 c:control_collection) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:56839
   [junit4]   2> 1288574 INFO  (coreZkRegister-2762-thread-1-processing-s:shard1 r:core_node1 n:127.0.0.1:56839_ x:collection1 c:control_collection) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 1288574 INFO  (coreZkRegister-2762-thread-1-processing-s:shard1 r:core_node1 n:127.0.0.1:56839_ x:collection1 c:control_collection) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 1288574 INFO  (coreZkRegister-2762-thread-1-processing-s:shard1 r:core_node1 n:127.0.0.1:56839_ x:collection1 c:control_collection) [n:127.0.0.1:56839_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1288576 INFO  (OverseerStateUpdate-94747307604443142-127.0.0.1:56839_-n_0000000000) [n:127.0.0.1:56839_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "base_url":"http://127.0.0.1:56839",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "node_name":"127.0.0.1:56839_",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "operation":"state"} current state version: 3
   [junit4]   2> 1288576 INFO  (OverseerStateUpdate-94747307604443142-127.0.0.1:56839_-n_0000000000) [n:127.0.0.1:56839_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "base_url":"http://127.0.0.1:56839",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "node_name":"127.0.0.1:56839_",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 1288679 INFO  (zkCallback-1007-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1288739 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1288740 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1288742 INFO  (zkCallback-1009-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5faaf408 name:ZooKeeperConnection Watcher:127.0.0.1:44976/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1288742 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1288742 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1288742 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1288745 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 1289054 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_6D2393130133B000-001/shard-1-001/cores/collection1
   [junit4]   2> 1289055 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_6D2393130133B000-001/shard-1-001
   [junit4]   2> 1289056 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 1289058 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@66a67b1{/,null,AVAILABLE}
   [junit4]   2> 1289059 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.e.j.s.ServerConnector Started ServerConnector@723cd397{HTTP/1.1}{127.0.0.1:38621}
   [junit4]   2> 1289059 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.e.j.s.Server Started @1291961ms
   [junit4]   2> 1289059 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_6D2393130133B000-001/tempDir-001/jetty1, hostContext=/, solrconfig=solrconfig.xml, hostPort=38621, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_6D2393130133B000-001/shard-1-001/cores}
   [junit4]   2> 1289059 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@6b482747
   [junit4]   2> 1289059 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_6D2393130133B000-001/shard-1-001/'
   [junit4]   2> 1289059 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1289059 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1289076 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1289077 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1289079 INFO  (zkCallback-1010-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@570481dc name:ZooKeeperConnection Watcher:127.0.0.1:44976/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1289079 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1289079 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1289080 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1289080 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_6D2393130133B000-001/shard-1-001/solr.xml
   [junit4]   2> 1289096 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_6D2393130133B000-001/shard-1-001/cores
   [junit4]   2> 1289096 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.CoreContainer New CoreContainer 1051817571
   [junit4]   2> 1289096 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_6D2393130133B000-001/shard-1-001/]
   [junit4]   2> 1289096 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_6D2393130133B000-001/shard-1-001/lib
   [junit4]   2> 1289096 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_6D2393130133B000-001/shard-1-001/lib).
   [junit4]   2> 1289104 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1289105 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1289106 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1289106 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1289106 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:44976/solr
   [junit4]   2> 1289106 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 1289106 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1289106 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1289108 INFO  (zkCallback-1012-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@708c513d name:ZooKeeperConnection Watcher:127.0.0.1:44976 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1289108 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1289108 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1289110 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [n:127.0.0.1:38621_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1289111 INFO  (zkCallback-1013-thread-1-processing-n:127.0.0.1:38621_) [n:127.0.0.1:38621_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@22ff011f name:ZooKeeperConnection Watcher:127.0.0.1:44976/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1289111 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [n:127.0.0.1:38621_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1289118 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [n:127.0.0.1:38621_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1290124 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [n:127.0.0.1:38621_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:38621_
   [junit4]   2> 1290125 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[6D2393130133B000]) [n:127.0.0.1:38621_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:38621_
   [junit4]   2> 1290126 INFO  (zkCallback-1007-thread-1-processing-n:127.0.0.1:56839_) [n:127.0.0.1:56839_    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1290126 INFO  (zkCallback-1013-thread-1-processing-n:127.0.0.1:38621_) [n:127.0.0.1:38621_    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected 

[...truncated too long message...]

ildren watches above limit:
   [junit4]   2> 
   [junit4]   2> 	6	/solr/collections
   [junit4]   2> 	4	/solr/live_nodes
   [junit4]   2> 	2	/solr/overseer/queue-work
   [junit4]   2> 	2	/solr/overseer/queue
   [junit4]   2> 	2	/solr/overseer/collection-queue-work
   [junit4]   2> 
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=LeaderInitiatedRecoveryOnShardRestartTest -Dtests.method=testRestartWithAllInLIR -Dtests.seed=6D2393130133B000 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=no_NO_NY -Dtests.timezone=America/Halifax -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
   [junit4] ERROR    367s J2 | LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR <<<
   [junit4]    > Throwable #1: java.lang.AssertionError: There are still nodes recoverying - waited for 330 seconds
   [junit4]    > 	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:172)
   [junit4]    > 	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:133)
   [junit4]    > 	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:128)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.waitForRecoveriesToFinish(AbstractFullDistribZkTestBase.java:830)
   [junit4]    > 	at org.apache.solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR(LeaderInitiatedRecoveryOnShardRestartTest.java:81)
   [junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:963)
   [junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:938)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)Throwable #2: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=20121, name=coreZkRegister-6242-thread-1, state=RUNNABLE, group=TGRP-LeaderInitiatedRecoveryOnShardRestartTest]
   [junit4]    > Caused by: java.lang.AssertionError
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([6D2393130133B000]:0)
   [junit4]    > 	at org.apache.solr.cloud.ZkController.updateLeaderInitiatedRecoveryState(ZkController.java:2133)
   [junit4]    > 	at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:434)
   [junit4]    > 	at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)
   [junit4]    > 	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)
   [junit4]    > 	at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:346)
   [junit4]    > 	at org.apache.solr.cloud.ZkController.joinElection(ZkController.java:1113)
   [junit4]    > 	at org.apache.solr.cloud.ZkController.register(ZkController.java:926)
   [junit4]    > 	at org.apache.solr.cloud.ZkController.register(ZkController.java:881)
   [junit4]    > 	at org.apache.solr.core.ZkContainer$2.run(ZkContainer.java:183)
   [junit4]    > 	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)
   [junit4]    > 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   [junit4]    > 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 3452655 INFO  (SUITE-LeaderInitiatedRecoveryOnShardRestartTest-seed#[6D2393130133B000]-worker) [n:127.0.0.1:50123_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_6D2393130133B000-001
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene53): {}, docValues:{}, sim=DefaultSimilarity, locale=no_NO_NY, timezone=America/Halifax
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_72 (64-bit)/cpus=4,threads=1,free=288424312,total=526909440
   [junit4]   2> NOTE: All tests run in this JVM: [BinaryUpdateRequestHandlerTest, DistributedFacetPivotLargeTest, CachingDirectoryFactoryTest, TestConfigSetsAPIZkFailure, TestCloudManagedSchemaConcurrent, TestJmxIntegration, TestOrdValues, TestDynamicFieldCollectionResource, TestReRankQParserPlugin, TestReloadAndDeleteDocs, SuggesterTest, TestHashQParserPlugin, QueryResultKeyTest, TestSimpleTrackingShardHandler, DocValuesMultiTest, TestScoreJoinQPScore, TestRequestStatusCollectionAPI, TestRebalanceLeaders, SolrCoreCheckLockOnStartupTest, TestDefaultSearchFieldResource, BasicDistributedZk2Test, FieldMutatingUpdateProcessorTest, TestReload, DateRangeFieldTest, TestReversedWildcardFilterFactory, TestNRTOpen, TestElisionMultitermQuery, QueryParsingTest, TestXIncludeConfig, CurrencyFieldXmlFileTest, TestLRUCache, CollectionsAPIAsyncDistributedZkTest, BlockDirectoryTest, TestWriterPerf, RulesTest, TestBulkSchemaAPI, StatsComponentTest, FullSolrCloudDistribCmdsTest, DocExpirationUpdateProcessorFactoryTest, TestFuzzyAnalyzedSuggestions, StatelessScriptUpdateProcessorFactoryTest, DistributedSuggestComponentTest, TestSystemIdResolver, InfoHandlerTest, TestSort, RAMDirectoryFactoryTest, TestExceedMaxTermLength, BadComponentTest, HighlighterConfigTest, LeaderElectionTest, TestRemoveLastDynamicCopyField, AliasIntegrationTest, MergeStrategyTest, TestConfigSetImmutable, SampleTest, UUIDFieldTest, TestSmileRequest, TriLevelCompositeIdRoutingTest, AssignTest, AutoCommitTest, SparseHLLTest, TestLeaderElectionZkExpiry, DebugComponentTest, ScriptEngineTest, SuggesterTSTTest, TestSolrConfigHandlerConcurrent, XmlUpdateRequestHandlerTest, RuleEngineTest, DistributedQueryComponentCustomSortTest, OverseerRolesTest, TestSolrQueryParser, LukeRequestHandlerTest, DistributedExpandComponentTest, TestIntervalFaceting, ConcurrentDeleteAndCreateCollectionTest, DistribDocExpirationUpdateProcessorTest, FieldAnalysisRequestHandlerTest, TestChildDocTransformer, DirectUpdateHandlerTest, TestFastWriter, ShowFileRequestHandlerTest, TestEmbeddedSolrServerConstructors, UpdateParamsTest, TestFieldTypeCollectionResource, TestSolrIndexConfig, SearchHandlerTest, TestReplicaProperties, SimpleFacetsTest, ChaosMonkeySafeLeaderTest, UnloadDistributedZkTest, SyncSliceTest, OpenCloseCoreStressTest, OverseerTest, TestRecovery, DistributedSpellCheckComponentTest, TestStressReorder, TestStressVersions, TestCoreContainer, TestSolr4Spatial, SolrCmdDistributorTest, PeerSyncTest, BadIndexSchemaTest, TestFunctionQuery, TestIndexSearcher, HighlighterTest, DistributedQueryElevationComponentTest, AnalysisAfterCoreReloadTest, TestExtendedDismaxParser, DocValuesTest, TestTrie, SpatialFilterTest, NoCacheHeaderTest, QueryEqualityTest, SortByFunctionTest, DistanceFunctionTest, SolrInfoMBeanTest, DisMaxRequestHandlerTest, PrimitiveFieldTypeTest, TermsComponentTest, PathHierarchyTokenizerFactoryTest, TestIndexingPerformance, TestArbitraryIndexDir, TestCollationField, ReturnFieldsTest, JSONWriterTest, MBeansHandlerTest, UniqFieldsUpdateProcessorFactoryTest, CSVRequestHandlerTest, TestLFUCache, PreAnalyzedFieldTest, PrimUtilsTest, DOMUtilTest, ClusterStateTest, TestUtils, TestDocumentBuilder, ZkNodePropsTest, SystemInfoHandlerTest, FileUtilsTest, TestRTGBase, DistributedIntervalFacetingTest, CleanupOldIndexTest, CloudExitableDirectoryReaderTest, CollectionStateFormat2Test, CollectionTooManyReplicasTest, DeleteLastCustomShardedReplicaTest, DeleteShardTest, DistribCursorPagingTest, LeaderFailoverAfterPartitionTest, LeaderInitiatedRecoveryOnCommitTest, LeaderInitiatedRecoveryOnShardRestartTest]
   [junit4] Completed [433/543] on J2 in 366.80s, 1 test, 1 error <<< FAILURES!

[...truncated 355 lines...]
   [junit4] JVM J1: stdout was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/temp/junit4-J1-20151024_190452_256.sysout
   [junit4] >>> JVM J1: stdout (verbatim) ----
   [junit4] java.lang.OutOfMemoryError: GC overhead limit exceeded
   [junit4] Dumping heap to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/heapdumps/java_pid19857.hprof ...
   [junit4] Heap dump file created [675741330 bytes in 11.260 secs]
   [junit4] <<< JVM J1: EOF ----

[...truncated 15 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 543 suites (7 ignored), 2144 tests, 2 errors, 1 failure, 31 ignored (18 assumptions) [seed: 6D2393130133B000]

Total time: 191 minutes 39 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 995 - Still Failing

[Apache Jenkins Server <je...@builds.apache.org>]

Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/995/

5 tests failed.
FAILED:  org.apache.solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR

Error Message:
Captured an uncaught exception in thread: Thread[id=484, name=coreZkRegister-215-thread-1, state=RUNNABLE, group=TGRP-LeaderInitiatedRecoveryOnShardRestartTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=484, name=coreZkRegister-215-thread-1, state=RUNNABLE, group=TGRP-LeaderInitiatedRecoveryOnShardRestartTest]
Caused by: java.lang.AssertionError
	at __randomizedtesting.SeedInfo.seed([604EA955F55DFDE5]:0)
	at org.apache.solr.cloud.ZkController.updateLeaderInitiatedRecoveryState(ZkController.java:2133)
	at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:434)
	at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)
	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)
	at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:346)
	at org.apache.solr.cloud.ZkController.joinElection(ZkController.java:1113)
	at org.apache.solr.cloud.ZkController.register(ZkController.java:926)
	at org.apache.solr.cloud.ZkController.register(ZkController.java:881)
	at org.apache.solr.core.ZkContainer$2.run(ZkContainer.java:183)
	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  junit.framework.TestSuite.org.apache.solr.core.TestLazyCores

Error Message:
ERROR: SolrIndexSearcher opens=51 closes=50

Stack Trace:
java.lang.AssertionError: ERROR: SolrIndexSearcher opens=51 closes=50
	at __randomizedtesting.SeedInfo.seed([604EA955F55DFDE5]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.solr.SolrTestCaseJ4.endTrackingSearchers(SolrTestCaseJ4.java:468)
	at org.apache.solr.SolrTestCaseJ4.afterClass(SolrTestCaseJ4.java:234)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1665)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  junit.framework.TestSuite.org.apache.solr.core.TestLazyCores

Error Message:
file handle leaks: [FileChannel(/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/index-SimpleFSDirectory-009/write.lock)]

Stack Trace:
java.lang.RuntimeException: file handle leaks: [FileChannel(/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/index-SimpleFSDirectory-009/write.lock)]
	at org.apache.lucene.mockfile.LeakFS.onClose(LeakFS.java:64)
	at org.apache.lucene.mockfile.FilterFileSystem.close(FilterFileSystem.java:78)
	at org.apache.lucene.mockfile.FilterFileSystem.close(FilterFileSystem.java:79)
	at org.apache.lucene.util.TestRuleTemporaryFilesCleanup.afterAlways(TestRuleTemporaryFilesCleanup.java:230)
	at com.carrotsearch.randomizedtesting.rules.TestRuleAdapter$1.afterAlways(TestRuleAdapter.java:31)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:43)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.Exception
	at org.apache.lucene.mockfile.LeakFS.onOpen(LeakFS.java:47)
	at org.apache.lucene.mockfile.HandleTrackingFS.callOpenHook(HandleTrackingFS.java:82)
	at org.apache.lucene.mockfile.HandleTrackingFS.newFileChannel(HandleTrackingFS.java:198)
	at org.apache.lucene.mockfile.HandleTrackingFS.newFileChannel(HandleTrackingFS.java:167)
	at java.nio.channels.FileChannel.open(FileChannel.java:287)
	at java.nio.channels.FileChannel.open(FileChannel.java:334)
	at org.apache.lucene.store.NativeFSLockFactory.obtainFSLock(NativeFSLockFactory.java:113)
	at org.apache.lucene.store.FSLockFactory.obtainLock(FSLockFactory.java:41)
	at org.apache.lucene.store.BaseDirectory.obtainLock(BaseDirectory.java:45)
	at org.apache.lucene.store.FilterDirectory.obtainLock(FilterDirectory.java:94)
	at org.apache.lucene.store.MockDirectoryWrapper.obtainLock(MockDirectoryWrapper.java:985)
	at org.apache.lucene.store.SleepingLockWrapper.obtainLock(SleepingLockWrapper.java:84)
	at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:775)
	at org.apache.solr.update.SolrIndexWriter.<init>(SolrIndexWriter.java:78)
	at org.apache.solr.update.SolrIndexWriter.create(SolrIndexWriter.java:65)
	at org.apache.solr.update.DefaultSolrCoreState.createMainIndexWriter(DefaultSolrCoreState.java:231)
	at org.apache.solr.update.DefaultSolrCoreState.getIndexWriter(DefaultSolrCoreState.java:105)
	at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1642)
	at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1784)
	at org.apache.solr.core.SolrCore.initSearcher(SolrCore.java:927)
	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:804)
	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:672)
	at org.apache.solr.core.CoreContainer.create(CoreContainer.java:729)
	at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:449)
	at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:440)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	... 1 more


FAILED:  junit.framework.TestSuite.org.apache.solr.core.TestLazyCores

Error Message:
1 thread leaked from SUITE scope at org.apache.solr.core.TestLazyCores:     1) Thread[id=132, name=searcherExecutor-76-thread-1, state=WAITING, group=TGRP-TestLazyCores]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.core.TestLazyCores: 
   1) Thread[id=132, name=searcherExecutor-76-thread-1, state=WAITING, group=TGRP-TestLazyCores]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
	at __randomizedtesting.SeedInfo.seed([604EA955F55DFDE5]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.core.TestLazyCores

Error Message:
There are still zombie threads that couldn't be terminated:    1) Thread[id=132, name=searcherExecutor-76-thread-1, state=WAITING, group=TGRP-TestLazyCores]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   1) Thread[id=132, name=searcherExecutor-76-thread-1, state=WAITING, group=TGRP-TestLazyCores]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
	at __randomizedtesting.SeedInfo.seed([604EA955F55DFDE5]:0)




Build Log:
[...truncated 9891 lines...]
   [junit4] Suite: org.apache.solr.core.TestLazyCores
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/init-core-data-001
   [junit4]   2> 11770 INFO  (SUITE-TestLazyCores-seed#[604EA955F55DFDE5]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true)
   [junit4]   2> 11790 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testRace
   [junit4]   2> 11848 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/'
   [junit4]   2> 11848 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 11848 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 11875 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    ] o.a.s.c.CoreContainer New CoreContainer 982569151
   [junit4]   2> 11875 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/]
   [junit4]   2> 11875 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/lib
   [junit4]   2> 11875 WARN  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/lib).
   [junit4]   2> 11914 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 600000,connTimeout : 60000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 11928 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=600000&connTimeout=60000&retry=true
   [junit4]   2> 11929 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    ] o.a.s.c.CoreContainer Allocating transient cache for 4 transient cores
   [junit4]   2> 11931 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 11931 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 11931 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 11931 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 11933 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    ] o.a.s.c.SolrCore Created CoreDescriptor: {configSetProperties=configsetprops.json, schema=schema.xml, dataDir=data/, instanceDir=collection1, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection1/, transient=false, name=collection1, config=solrconfig.xml, loadOnStartup=true}
   [junit4]   2> 11933 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    ] o.a.s.c.SolrCore Created CoreDescriptor: {absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection2/, dataDir=data/, transient=true, config=solrconfig.xml, instanceDir=collection2, loadOnStartup=true, name=collectionLazy2, configSetProperties=configsetprops.json, schema=schema.xml}
   [junit4]   2> 11934 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    ] o.a.s.c.SolrCore Created CoreDescriptor: {dataDir=data/, config=solrconfig.xml, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection3/, configSetProperties=configsetprops.json, instanceDir=collection3, schema=schema.xml, loadOnStartup=false, transient=on, name=collectionLazy3}
   [junit4]   2> 11934 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    ] o.a.s.c.SolrCore Created CoreDescriptor: {instanceDir=collection4, name=collectionLazy4, loadOnStartup=false, configSetProperties=configsetprops.json, schema=schema.xml, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection4/, dataDir=data/, transient=false, config=solrconfig.xml}
   [junit4]   2> 11935 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    ] o.a.s.c.SolrCore Created CoreDescriptor: {instanceDir=collection5, schema=schema.xml, configSetProperties=configsetprops.json, transient=false, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection5/, config=solrconfig.xml, name=collectionLazy5, loadOnStartup=true, dataDir=data/}
   [junit4]   2> 11935 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    ] o.a.s.c.SolrCore Created CoreDescriptor: {name=collectionLazy6, instanceDir=collection6, transient=true, loadOnStartup=false, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection6/, configSetProperties=configsetprops.json, schema=schema.xml, config=solrconfig.xml, dataDir=data/}
   [junit4]   2> 11936 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    ] o.a.s.c.SolrCore Created CoreDescriptor: {name=collectionLazy7, schema=schema.xml, dataDir=data/, transient=true, configSetProperties=configsetprops.json, config=solrconfig.xml, loadOnStartup=false, instanceDir=collection7, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection7/}
   [junit4]   2> 11936 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    ] o.a.s.c.SolrCore Created CoreDescriptor: {name=collectionLazy8, configSetProperties=configsetprops.json, transient=true, schema=schema.xml, instanceDir=collection8, loadOnStartup=false, dataDir=data/, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection8/, config=solrconfig.xml}
   [junit4]   2> 11937 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    ] o.a.s.c.SolrCore Created CoreDescriptor: {config=solrconfig.xml, transient=true, schema=schema.xml, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection9/, dataDir=data/, configSetProperties=configsetprops.json, name=collectionLazy9, loadOnStartup=false, instanceDir=collection9}
   [junit4]   2> 11938 INFO  (coreLoadExecutor-27-thread-1) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection1/'
   [junit4]   2> 11942 INFO  (coreLoadExecutor-27-thread-1) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 11942 INFO  (coreLoadExecutor-27-thread-1) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 11947 INFO  (coreLoadExecutor-27-thread-3) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection5/'
   [junit4]   2> 11949 INFO  (coreLoadExecutor-27-thread-2) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection2/'
   [junit4]   2> 11949 INFO  (coreLoadExecutor-27-thread-2) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 11949 INFO  (coreLoadExecutor-27-thread-2) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 11954 INFO  (coreLoadExecutor-27-thread-3) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 11954 INFO  (coreLoadExecutor-27-thread-3) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 12045 INFO  (coreLoadExecutor-27-thread-1) [    ] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 12053 INFO  (coreLoadExecutor-27-thread-3) [    ] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 12079 INFO  (coreLoadExecutor-27-thread-2) [    ] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 12103 INFO  (coreLoadExecutor-27-thread-1) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 12138 INFO  (coreLoadExecutor-27-thread-2) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 12150 INFO  (coreLoadExecutor-27-thread-3) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 12241 INFO  (coreLoadExecutor-27-thread-1) [    ] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 12241 INFO  (coreLoadExecutor-27-thread-1) [    ] o.a.s.s.IndexSchema Reading Solr Schema from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection1/conf/schema.xml
   [junit4]   2> 12250 INFO  (coreLoadExecutor-27-thread-2) [    ] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 12251 INFO  (coreLoadExecutor-27-thread-2) [    ] o.a.s.s.IndexSchema Reading Solr Schema from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection2/conf/schema.xml
   [junit4]   2> 12267 INFO  (coreLoadExecutor-27-thread-2) [    ] o.a.s.s.IndexSchema [collectionLazy2] Schema name=tiny
   [junit4]   2> 12299 INFO  (coreLoadExecutor-27-thread-1) [    ] o.a.s.s.IndexSchema [collection1] Schema name=tiny
   [junit4]   2> 12337 INFO  (coreLoadExecutor-27-thread-2) [    ] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 12339 INFO  (coreLoadExecutor-27-thread-3) [    ] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 12339 INFO  (coreLoadExecutor-27-thread-3) [    ] o.a.s.s.IndexSchema Reading Solr Schema from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection5/conf/schema.xml
   [junit4]   2> 12351 INFO  (coreLoadExecutor-27-thread-1) [    ] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 12351 INFO  (coreLoadExecutor-27-thread-3) [    ] o.a.s.s.IndexSchema [collectionLazy5] Schema name=tiny
   [junit4]   2> 12353 INFO  (coreLoadExecutor-27-thread-1) [    ] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection1/conf'
   [junit4]   2> 12353 INFO  (coreLoadExecutor-27-thread-1) [    ] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection1/
   [junit4]   2> 12353 INFO  (coreLoadExecutor-27-thread-1) [    x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 12379 INFO  (coreLoadExecutor-27-thread-1) [    x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection1/], dataDir=[null]
   [junit4]   2> 12379 INFO  (coreLoadExecutor-27-thread-1) [    x:collection1] o.a.s.c.SolrCore JMX monitoring not detected for core: collection1
   [junit4]   2> 12380 INFO  (coreLoadExecutor-27-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection1/data
   [junit4]   2> 12380 INFO  (coreLoadExecutor-27-thread-1) [    x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection1/data/index/
   [junit4]   2> 12380 WARN  (coreLoadExecutor-27-thread-1) [    x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 12388 INFO  (coreLoadExecutor-27-thread-2) [    ] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection2/conf'
   [junit4]   2> 12388 INFO  (coreLoadExecutor-27-thread-2) [    ] o.a.s.c.CoreContainer Creating SolrCore 'collectionLazy2' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection2/
   [junit4]   2> 12388 INFO  (coreLoadExecutor-27-thread-2) [    x:collectionLazy2] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 12389 INFO  (coreLoadExecutor-27-thread-2) [    x:collectionLazy2] o.a.s.c.SolrCore [[collectionLazy2] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection2/], dataDir=[null]
   [junit4]   2> 12389 INFO  (coreLoadExecutor-27-thread-2) [    x:collectionLazy2] o.a.s.c.SolrCore JMX monitoring not detected for core: collectionLazy2
   [junit4]   2> 12400 INFO  (coreLoadExecutor-27-thread-2) [    x:collectionLazy2] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection2/data
   [junit4]   2> 12401 INFO  (coreLoadExecutor-27-thread-2) [    x:collectionLazy2] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection2/data/index/
   [junit4]   2> 12401 WARN  (coreLoadExecutor-27-thread-2) [    x:collectionLazy2] o.a.s.c.SolrCore [collectionLazy2] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection2/data/index' doesn't exist. Creating new index...
   [junit4]   2> 12407 INFO  (coreLoadExecutor-27-thread-2) [    x:collectionLazy2] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection2/data/index
   [junit4]   2> 12408 INFO  (coreLoadExecutor-27-thread-2) [    x:collectionLazy2] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=3, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.3906092116355694]
   [junit4]   2> 12401 INFO  (coreLoadExecutor-27-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection1/data/index
   [junit4]   2> 12408 INFO  (coreLoadExecutor-27-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=3, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.3906092116355694]
   [junit4]   2> 12408 INFO  (coreLoadExecutor-27-thread-3) [    ] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 12410 INFO  (coreLoadExecutor-27-thread-3) [    ] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection5/conf'
   [junit4]   2> 12410 INFO  (coreLoadExecutor-27-thread-3) [    ] o.a.s.c.CoreContainer Creating SolrCore 'collectionLazy5' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection5/
   [junit4]   2> 12410 INFO  (coreLoadExecutor-27-thread-3) [    x:collectionLazy5] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 12411 INFO  (coreLoadExecutor-27-thread-3) [    x:collectionLazy5] o.a.s.c.SolrCore [[collectionLazy5] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection5/], dataDir=[null]
   [junit4]   2> 12411 INFO  (coreLoadExecutor-27-thread-3) [    x:collectionLazy5] o.a.s.c.SolrCore JMX monitoring not detected for core: collectionLazy5
   [junit4]   2> 12414 INFO  (coreLoadExecutor-27-thread-3) [    x:collectionLazy5] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection5/data
   [junit4]   2> 12414 INFO  (coreLoadExecutor-27-thread-3) [    x:collectionLazy5] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection5/data/index/
   [junit4]   2> 12414 WARN  (coreLoadExecutor-27-thread-3) [    x:collectionLazy5] o.a.s.c.SolrCore [collectionLazy5] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection5/data/index' doesn't exist. Creating new index...
   [junit4]   2> 12420 INFO  (coreLoadExecutor-27-thread-3) [    x:collectionLazy5] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection5/data/index
   [junit4]   2> 12420 INFO  (coreLoadExecutor-27-thread-3) [    x:collectionLazy5] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=3, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.3906092116355694]
   [junit4]   2> 12423 INFO  (coreLoadExecutor-27-thread-1) [    x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(SimpleFSDirectory@/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/index-SimpleFSDirectory-001 lockFactory=org.apache.lucene.store.NativeFSLockFactory@18be5a8),segFN=segments_1,generation=1}
   [junit4]   2> 12424 INFO  (coreLoadExecutor-27-thread-1) [    x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 12424 INFO  (coreLoadExecutor-27-thread-2) [    x:collectionLazy2] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(SimpleFSDirectory@/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/index-SimpleFSDirectory-002 lockFactory=org.apache.lucene.store.NativeFSLockFactory@18be5a8),segFN=segments_1,generation=1}
   [junit4]   2> 12424 INFO  (coreLoadExecutor-27-thread-2) [    x:collectionLazy2] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 12426 INFO  (coreLoadExecutor-27-thread-3) [    x:collectionLazy5] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(SimpleFSDirectory@/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/index-SimpleFSDirectory-003 lockFactory=org.apache.lucene.store.NativeFSLockFactory@18be5a8),segFN=segments_1,generation=1}
   [junit4]   2> 12426 INFO  (coreLoadExecutor-27-thread-3) [    x:collectionLazy5] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 12436 INFO  (OldIndexDirectoryCleanupThreadForCore-collectionLazy2) [    x:collectionLazy2] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collectionLazy2 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection2/data/
   [junit4]   2> 12437 WARN  (OldIndexDirectoryCleanupThreadForCore-collectionLazy2) [    x:collectionLazy2] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection2/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 12453 INFO  (coreLoadExecutor-27-thread-2) [    x:collectionLazy2] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 12453 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [    x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection1/data/
   [junit4]   2> 12454 INFO  (OldIndexDirectoryCleanupThreadForCore-collectionLazy5) [    x:collectionLazy5] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collectionLazy5 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection5/data/
   [junit4]   2> 12457 INFO  (coreLoadExecutor-27-thread-1) [    x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 12458 INFO  (coreLoadExecutor-27-thread-3) [    x:collectionLazy5] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 12460 INFO  (coreLoadExecutor-27-thread-3) [    x:collectionLazy5] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 12460 INFO  (coreLoadExecutor-27-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 12454 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [    x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 12454 WARN  (OldIndexDirectoryCleanupThreadForCore-collectionLazy5) [    x:collectionLazy5] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection5/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 12463 INFO  (coreLoadExecutor-27-thread-2) [    x:collectionLazy2] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 12463 INFO  (coreLoadExecutor-27-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 12465 INFO  (coreLoadExecutor-27-thread-2) [    x:collectionLazy2] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 12465 INFO  (coreLoadExecutor-27-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 12467 INFO  (coreLoadExecutor-27-thread-2) [    x:collectionLazy2] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 12468 INFO  (coreLoadExecutor-27-thread-2) [    x:collectionLazy2] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 12476 INFO  (coreLoadExecutor-27-thread-3) [    x:collectionLazy5] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 12482 INFO  (coreLoadExecutor-27-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 12490 INFO  (coreLoadExecutor-27-thread-3) [    x:collectionLazy5] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 12492 INFO  (coreLoadExecutor-27-thread-3) [    x:collectionLazy5] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 12503 INFO  (coreLoadExecutor-27-thread-1) [    x:collection1] o.a.s.c.RequestHandlers Registered paths: /update,/update/json/docs,/admin/logging,/admin/plugins,/admin/mbeans,/admin/segments,/admin/ping,/admin/luke,/config,/replication,/admin/file,/admin/properties,/schema,/admin/threads,/get,/update/json,/select,/admin/system,/update/csv
   [junit4]   2> 12505 INFO  (coreLoadExecutor-27-thread-1) [    x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 12518 INFO  (coreLoadExecutor-27-thread-1) [    x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 12518 INFO  (coreLoadExecutor-27-thread-1) [    x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 12520 INFO  (coreLoadExecutor-27-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=9, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.3284026821269601]
   [junit4]   2> 12537 INFO  (coreLoadExecutor-27-thread-2) [    x:collectionLazy2] o.a.s.c.RequestHandlers Registered paths: /update,/update/json/docs,/admin/logging,/admin/plugins,/admin/mbeans,/admin/segments,/admin/ping,/admin/luke,/config,/replication,/admin/file,/admin/properties,/schema,/admin/threads,/get,/update/json,/select,/admin/system,/update/csv
   [junit4]   2> 12538 INFO  (coreLoadExecutor-27-thread-3) [    x:collectionLazy5] o.a.s.c.RequestHandlers Registered paths: /update,/update/json/docs,/admin/logging,/admin/plugins,/admin/mbeans,/admin/segments,/admin/ping,/admin/luke,/config,/replication,/admin/file,/admin/properties,/schema,/admin/threads,/get,/update/json,/select,/admin/system,/update/csv
   [junit4]   2> 12539 INFO  (coreLoadExecutor-27-thread-2) [    x:collectionLazy2] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 12540 INFO  (coreLoadExecutor-27-thread-2) [    x:collectionLazy2] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 12540 INFO  (coreLoadExecutor-27-thread-2) [    x:collectionLazy2] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 12540 INFO  (coreLoadExecutor-27-thread-2) [    x:collectionLazy2] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=9, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.3284026821269601]
   [junit4]   2> 12545 INFO  (coreLoadExecutor-27-thread-3) [    x:collectionLazy5] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 12547 INFO  (coreLoadExecutor-27-thread-3) [    x:collectionLazy5] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 12547 INFO  (coreLoadExecutor-27-thread-3) [    x:collectionLazy5] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 12547 INFO  (coreLoadExecutor-27-thread-3) [    x:collectionLazy5] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=9, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.3284026821269601]
   [junit4]   2> 12593 INFO  (coreLoadExecutor-27-thread-2) [    x:collectionLazy2] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(SimpleFSDirectory@/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/index-SimpleFSDirectory-002 lockFactory=org.apache.lucene.store.NativeFSLockFactory@18be5a8),segFN=segments_1,generation=1}
   [junit4]   2> 12593 INFO  (coreLoadExecutor-27-thread-2) [    x:collectionLazy2] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 12593 INFO  (coreLoadExecutor-27-thread-1) [    x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(SimpleFSDirectory@/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/index-SimpleFSDirectory-001 lockFactory=org.apache.lucene.store.NativeFSLockFactory@18be5a8),segFN=segments_1,generation=1}
   [junit4]   2> 12593 INFO  (coreLoadExecutor-27-thread-1) [    x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 12593 INFO  (coreLoadExecutor-27-thread-2) [    x:collectionLazy2] o.a.s.s.SolrIndexSearcher Opening Searcher@64d2da85[collectionLazy2] main
   [junit4]   2> 12593 INFO  (coreLoadExecutor-27-thread-1) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@3872f378[collection1] main
   [junit4]   2> 12594 INFO  (coreLoadExecutor-27-thread-2) [    x:collectionLazy2] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection2/conf
   [junit4]   2> 12594 INFO  (coreLoadExecutor-27-thread-2) [    x:collectionLazy2] o.a.s.r.RestManager Initializing RestManager with initArgs: {storageDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection2/conf}
   [junit4]   2> 12594 INFO  (coreLoadExecutor-27-thread-2) [    x:collectionLazy2] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using file:dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection2/conf
   [junit4]   2> 12594 INFO  (coreLoadExecutor-27-thread-2) [    x:collectionLazy2] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 12594 INFO  (coreLoadExecutor-27-thread-2) [    x:collectionLazy2] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 12595 INFO  (coreLoadExecutor-27-thread-2) [    x:collectionLazy2] o.a.s.c.CoreContainer Opening transient core collectionLazy2
   [junit4]   2> 12595 INFO  (coreLoadExecutor-27-thread-2) [    x:collectionLazy2] o.a.s.c.CoreContainer registering core: collectionLazy2
   [junit4]   2> 12596 INFO  (searcherExecutor-29-thread-1-processing-x:collectionLazy2) [    x:collectionLazy2] o.a.s.c.SolrCore [collectionLazy2] Registered new searcher Searcher@64d2da85[collectionLazy2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 12594 INFO  (coreLoadExecutor-27-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection1/conf
   [junit4]   2> 12596 INFO  (coreLoadExecutor-27-thread-1) [    x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {storageDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection1/conf}
   [junit4]   2> 12596 INFO  (coreLoadExecutor-27-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using file:dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection1/conf
   [junit4]   2> 12596 INFO  (coreLoadExecutor-27-thread-1) [    x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 12596 INFO  (coreLoadExecutor-27-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 12597 INFO  (coreLoadExecutor-27-thread-1) [    x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 12600 INFO  (searcherExecutor-28-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@3872f378[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 12602 INFO  (coreLoadExecutor-27-thread-3) [    x:collectionLazy5] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(SimpleFSDirectory@/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/index-SimpleFSDirectory-003 lockFactory=org.apache.lucene.store.NativeFSLockFactory@18be5a8),segFN=segments_1,generation=1}
   [junit4]   2> 12602 INFO  (coreLoadExecutor-27-thread-3) [    x:collectionLazy5] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 12602 INFO  (coreLoadExecutor-27-thread-3) [    x:collectionLazy5] o.a.s.s.SolrIndexSearcher Opening Searcher@4a7437e4[collectionLazy5] main
   [junit4]   2> 12603 INFO  (coreLoadExecutor-27-thread-3) [    x:collectionLazy5] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection5/conf
   [junit4]   2> 12603 INFO  (coreLoadExecutor-27-thread-3) [    x:collectionLazy5] o.a.s.r.RestManager Initializing RestManager with initArgs: {storageDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection5/conf}
   [junit4]   2> 12603 INFO  (coreLoadExecutor-27-thread-3) [    x:collectionLazy5] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using file:dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection5/conf
   [junit4]   2> 12603 INFO  (coreLoadExecutor-27-thread-3) [    x:collectionLazy5] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 12603 INFO  (coreLoadExecutor-27-thread-3) [    x:collectionLazy5] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 12604 INFO  (coreLoadExecutor-27-thread-3) [    x:collectionLazy5] o.a.s.c.CoreContainer registering core: collectionLazy5
   [junit4]   2> 12604 INFO  (searcherExecutor-30-thread-1-processing-x:collectionLazy5) [    x:collectionLazy5] o.a.s.c.SolrCore [collectionLazy5] Registered new searcher Searcher@4a7437e4[collectionLazy5] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 12634 INFO  (Thread-14) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection3/'
   [junit4]   2> 12634 INFO  (Thread-14) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 12634 INFO  (Thread-14) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 12729 INFO  (Thread-14) [    ] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 12754 INFO  (Thread-14) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 12885 INFO  (Thread-14) [    ] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 12885 INFO  (Thread-14) [    ] o.a.s.s.IndexSchema Reading Solr Schema from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection3/conf/schema.xml
   [junit4]   2> 12894 INFO  (Thread-14) [    ] o.a.s.s.IndexSchema [collectionLazy3] Schema name=tiny
   [junit4]   2> 12912 INFO  (Thread-14) [    ] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 12914 INFO  (Thread-14) [    ] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection3/conf'
   [junit4]   2> 12914 INFO  (Thread-14) [    ] o.a.s.c.CoreContainer Creating SolrCore 'collectionLazy3' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection3/
   [junit4]   2> 12914 INFO  (Thread-14) [    x:collectionLazy3] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 12914 INFO  (Thread-14) [    x:collectionLazy3] o.a.s.c.SolrCore [[collectionLazy3] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection3/], dataDir=[null]
   [junit4]   2> 12914 INFO  (Thread-14) [    x:collectionLazy3] o.a.s.c.SolrCore JMX monitoring not detected for core: collectionLazy3
   [junit4]   2> 12915 INFO  (Thread-14) [    x:collectionLazy3] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection3/data
   [junit4]   2> 12915 INFO  (Thread-14) [    x:collectionLazy3] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection3/data/index/
   [junit4]   2> 12915 WARN  (Thread-14) [    x:collectionLazy3] o.a.s.c.SolrCore [collectionLazy3] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection3/data/index' doesn't exist. Creating new index...
   [junit4]   2> 12916 INFO  (Thread-14) [    x:collectionLazy3] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection3/data/index
   [junit4]   2> 12917 INFO  (Thread-14) [    x:collectionLazy3] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=3, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.3906092116355694]
   [junit4]   2> 12920 INFO  (Thread-14) [    x:collectionLazy3] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(SimpleFSDirectory@/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/index-SimpleFSDirectory-004 lockFactory=org.apache.lucene.store.NativeFSLockFactory@18be5a8),segFN=segments_1,generation=1}
   [junit4]   2> 12920 INFO  (Thread-14) [    x:collectionLazy3] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 12937 INFO  (Thread-14) [    x:collectionLazy3] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 12940 INFO  (Thread-14) [    x:collectionLazy3] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 12942 INFO  (Thread-14) [    x:collectionLazy3] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 12944 INFO  (Thread-14) [    x:collectionLazy3] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 12945 INFO  (OldIndexDirectoryCleanupThreadForCore-collectionLazy3) [    x:collectionLazy3] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collectionLazy3 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection3/data/
   [junit4]   2> 12946 WARN  (OldIndexDirectoryCleanupThreadForCore-collectionLazy3) [    x:collectionLazy3] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection3/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 12948 INFO  (Thread-14) [    x:collectionLazy3] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 12958 INFO  (Thread-14) [    x:collectionLazy3] o.a.s.c.RequestHandlers Registered paths: /update,/update/json/docs,/admin/logging,/admin/plugins,/admin/mbeans,/admin/segments,/admin/ping,/admin/luke,/config,/replication,/admin/file,/admin/properties,/schema,/admin/threads,/get,/update/json,/select,/admin/system,/update/csv
   [junit4]   2> 12961 INFO  (Thread-14) [    x:collectionLazy3] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 12961 INFO  (Thread-14) [    x:collectionLazy3] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 12961 INFO  (Thread-14) [    x:collectionLazy3] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 12961 INFO  (Thread-14) [    x:collectionLazy3] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=9, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.3284026821269601]
   [junit4]   2> 13014 INFO  (Thread-14) [    x:collectionLazy3] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(SimpleFSDirectory@/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/index-SimpleFSDirectory-004 lockFactory=org.apache.lucene.store.NativeFSLockFactory@18be5a8),segFN=segments_1,generation=1}
   [junit4]   2> 13014 INFO  (Thread-14) [    x:collectionLazy3] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 13015 INFO  (Thread-14) [    x:collectionLazy3] o.a.s.s.SolrIndexSearcher Opening Searcher@1886ad24[collectionLazy3] main
   [junit4]   2> 13015 INFO  (Thread-14) [    x:collectionLazy3] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection3/conf
   [junit4]   2> 13015 INFO  (Thread-14) [    x:collectionLazy3] o.a.s.r.RestManager Initializing RestManager with initArgs: {storageDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection3/conf}
   [junit4]   2> 13015 INFO  (Thread-14) [    x:collectionLazy3] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using file:dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection3/conf
   [junit4]   2> 13015 INFO  (Thread-14) [    x:collectionLazy3] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 13016 INFO  (Thread-14) [    x:collectionLazy3] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 13016 INFO  (Thread-14) [    x:collectionLazy3] o.a.s.c.CoreContainer Opening transient core collectionLazy3
   [junit4]   2> 13016 INFO  (Thread-14) [    x:collectionLazy3] o.a.s.c.CoreContainer registering core: collectionLazy3
   [junit4]   2> 13017 INFO  (searcherExecutor-40-thread-1-processing-x:collectionLazy3) [    x:collectionLazy3] o.a.s.c.SolrCore [collectionLazy3] Registered new searcher Searcher@1886ad24[collectionLazy3] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 13019 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=982569151
   [junit4]   2> 13019 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    x:collection1] o.a.s.c.SolrCore [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@1b906ce8
   [junit4]   2> 13020 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    x:collection1] o.a.s.u.UpdateHandler closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
   [junit4]   2> 13020 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    x:collection1] o.a.s.u.SolrCoreState Closing SolrCoreState
   [junit4]   2> 13020 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    x:collection1] o.a.s.u.DefaultSolrCoreState SolrCoreState ref count has reached 0 - closing IndexWriter
   [junit4]   2> 13020 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    x:collection1] o.a.s.u.DefaultSolrCoreState closing IndexWriter with IndexWriterCloser
   [junit4]   2> 13022 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    x:collection1] o.a.s.c.SolrCore [collection1] Closing main searcher on request.
   [junit4]   2> 13022 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing MockDirectoryFactory - 2 directories currently being tracked
   [junit4]   2> 13022 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    x:collection1] o.a.s.c.CachingDirectoryFactory looking to close /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection1/data/index [CachedDir<<refCount=0;path=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection1/data/index;done=false>>]
   [junit4]   2> 13022 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection1/data/index
   [junit4]   2> 13022 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    x:collection1] o.a.s.c.CachingDirectoryFactory looking to close /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection1/data [CachedDir<<refCount=0;path=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection1/data;done=false>>]
   [junit4]   2> 13022 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection1/data
   [junit4]   2> 13022 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    x:collectionLazy5] o.a.s.c.SolrCore [collectionLazy5]  CLOSING SolrCore org.apache.solr.core.SolrCore@6cec911f
   [junit4]   2> 13023 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    x:collectionLazy5] o.a.s.u.UpdateHandler closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
   [junit4]   2> 13023 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    x:collectionLazy5] o.a.s.u.SolrCoreState Closing SolrCoreState
   [junit4]   2> 13023 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    x:collectionLazy5] o.a.s.u.DefaultSolrCoreState SolrCoreState ref count has reached 0 - closing IndexWriter
   [junit4]   2> 13023 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    x:collectionLazy5] o.a.s.u.DefaultSolrCoreState closing IndexWriter with IndexWriterCloser
   [junit4]   2> 13029 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    x:collectionLazy5] o.a.s.c.SolrCore [collectionLazy5] Closing main searcher on request.
   [junit4]   2> 13030 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    x:collectionLazy5] o.a.s.c.CachingDirectoryFactory Closing MockDirectoryFactory - 2 directories currently being tracked
   [junit4]   2> 13030 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    x:collectionLazy5] o.a.s.c.CachingDirectoryFactory looking to close /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection5/data [CachedDir<<refCount=0;path=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection5/data;done=false>>]
   [junit4]   2> 13030 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    x:collectionLazy5] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection5/data
   [junit4]   2> 13030 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    x:collectionLazy5] o.a.s.c.CachingDirectoryFactory looking to close /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection5/data/index [CachedDir<<refCount=0;path=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection5/data/index;done=false>>]
   [junit4]   2> 13030 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    x:collectionLazy5] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection5/data/index
   [junit4]   2> 13030 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    x:collectionLazy2] o.a.s.c.SolrCore [collectionLazy2]  CLOSING SolrCore org.apache.solr.core.SolrCore@34105c79
   [junit4]   2> 13031 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    x:collectionLazy2] o.a.s.u.UpdateHandler closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
   [junit4]   2> 13031 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    x:collectionLazy2] o.a.s.u.SolrCoreState Closing SolrCoreState
   [junit4]   2> 13031 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    x:collectionLazy2] o.a.s.u.DefaultSolrCoreState SolrCoreState ref count has reached 0 - closing IndexWriter
   [junit4]   2> 13031 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    x:collectionLazy2] o.a.s.u.DefaultSolrCoreState closing IndexWriter with IndexWriterCloser
   [junit4]   2> 13033 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    x:collectionLazy2] o.a.s.c.SolrCore [collectionLazy2] Closing main searcher on request.
   [junit4]   2> 13033 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    x:collectionLazy2] o.a.s.c.CachingDirectoryFactory Closing MockDirectoryFactory - 2 directories currently being tracked
   [junit4]   2> 13033 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    x:collectionLazy2] o.a.s.c.CachingDirectoryFactory looking to close /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection2/data/index [CachedDir<<refCount=0;path=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection2/data/index;done=false>>]
   [junit4]   2> 13033 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    x:collectionLazy2] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection2/data/index
   [junit4]   2> 13033 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    x:collectionLazy2] o.a.s.c.CachingDirectoryFactory looking to close /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection2/data [CachedDir<<refCount=0;path=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection2/data;done=false>>]
   [junit4]   2> 13033 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    x:collectionLazy2] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection2/data
   [junit4]   2> 13034 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    x:collectionLazy3] o.a.s.c.SolrCore [collectionLazy3]  CLOSING SolrCore org.apache.solr.core.SolrCore@4ffc341f
   [junit4]   2> 13034 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    x:collectionLazy3] o.a.s.u.UpdateHandler closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
   [junit4]   2> 13034 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    x:collectionLazy3] o.a.s.u.SolrCoreState Closing SolrCoreState
   [junit4]   2> 13034 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    x:collectionLazy3] o.a.s.u.DefaultSolrCoreState SolrCoreState ref count has reached 0 - closing IndexWriter
   [junit4]   2> 13034 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    x:collectionLazy3] o.a.s.u.DefaultSolrCoreState closing IndexWriter with IndexWriterCloser
   [junit4]   2> 13035 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    x:collectionLazy3] o.a.s.c.SolrCore [collectionLazy3] Closing main searcher on request.
   [junit4]   2> 13036 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    x:collectionLazy3] o.a.s.c.CachingDirectoryFactory Closing MockDirectoryFactory - 2 directories currently being tracked
   [junit4]   2> 13036 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    x:collectionLazy3] o.a.s.c.CachingDirectoryFactory looking to close /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection3/data [CachedDir<<refCount=0;path=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection3/data;done=false>>]
   [junit4]   2> 13036 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    x:collectionLazy3] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection3/data
   [junit4]   2> 13036 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    x:collectionLazy3] o.a.s.c.CachingDirectoryFactory looking to close /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection3/data/index [CachedDir<<refCount=0;path=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection3/data/index;done=false>>]
   [junit4]   2> 13036 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    x:collectionLazy3] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-001/collection3/data/index
   [junit4]   2> 13036 INFO  (TEST-TestLazyCores.testRace-seed#[604EA955F55DFDE5]) [    x:collectionLazy3] o.a.s.SolrTestCaseJ4 ###Ending testRace
   [junit4]   2> 13059 INFO  (TEST-TestLazyCores.testLazySearch-seed#[604EA955F55DFDE5]) [    x:collectionLazy3] o.a.s.SolrTestCaseJ4 ###Starting testLazySearch
   [junit4]   2> 13128 INFO  (TEST-TestLazyCores.testLazySearch-seed#[604EA955F55DFDE5]) [    x:collectionLazy3] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-002/'
   [junit4]   2> 13128 INFO  (TEST-TestLazyCores.testLazySearch-seed#[604EA955F55DFDE5]) [    x:collectionLazy3] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 13128 INFO  (TEST-TestLazyCores.testLazySearch-seed#[604EA955F55DFDE5]) [    x:collectionLazy3] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 13151 INFO  (TEST-TestLazyCores.testLazySearch-seed#[604EA955F55DFDE5]) [    x:collectionLazy3] o.a.s.c.CoreContainer New CoreContainer 1680872867
   [junit4]   2> 13152 INFO  (TEST-TestLazyCores.testLazySearch-seed#[604EA955F55DFDE5]) [    x:collectionLazy3] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-002/]
   [junit4]   2> 13152 INFO  (TEST-TestLazyCores.testLazySearch-seed#[604EA955F55DFDE5]) [    x:collectionLazy3] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-002/lib
   [junit4]   2> 13152 WARN  (TEST-TestLazyCores.testLazySearch-seed#[604EA955F55DFDE5]) [    x:collectionLazy3] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-002/lib).
   [junit4]   2> 13165 INFO  (TEST-TestLazyCores.testLazySearch-seed#[604EA955F55DFDE5]) [    x:collectionLazy3] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 600000,connTimeout : 60000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 13167 INFO  (TEST-TestLazyCores.testLazySearch-seed#[604EA955F55DFDE5]) [    x:collectionLazy3] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=600000&connTimeout=60000&retry=true
   [junit4]   2> 13168 INFO  (TEST-TestLazyCores.testLazySearch-seed#[604EA955F55DFDE5]) [    x:collectionLazy3] o.a.s.c.CoreContainer Allocating transient cache for 4 transient cores
   [junit4]   2> 13168 INFO  (TEST-TestLazyCores.testLazySearch-seed#[604EA955F55DFDE5]) [    x:collectionLazy3] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 13168 INFO  (TEST-TestLazyCores.testLazySearch-seed#[604EA955F55DFDE5]) [    x:collectionLazy3] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 13168 INFO  (TEST-TestLazyCores.testLazySearch-seed#[604EA955F55DFDE5]) [    x:collectionLazy3] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 13168 INFO  (TEST-TestLazyCores.testLazySearch-seed#[604EA955F55DFDE5]) [    x:collectionLazy3] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 13170 INFO  (TEST-TestLazyCores.testLazySearch-seed#[604EA955F55DFDE5]) [    x:collectionLazy3] o.a.s.c.SolrCore Created CoreDescriptor: {schema=schema.xml, config=solrconfig.xml, dataDir=data/, instanceDir=collection1, loadOnStartup=true, name=collection1, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-002/collection1/, transient=false, configSetProperties=configsetprops.json}
   [junit4]   2> 13170 INFO  (TEST-TestLazyCores.testLazySearch-seed#[604EA955F55DFDE5]) [    x:collectionLazy3] o.a.s.c.SolrCore Created CoreDescriptor: {loadOnStartup=true, configSetProperties=configsetprops.json, name=collectionLazy2, schema=schema.xml, dataDir=data/, instanceDir=collection2, config=solrconfig.xml, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-002/collection2/, transient=true}
   [junit4]   2> 13171 INFO  (TEST-TestLazyCores.testLazySearch-seed#[604EA955F55DFDE5]) [    x:collectionLazy3] o.a.s.c.SolrCore Created CoreDescriptor: {dataDir=data/, config=solrconfig.xml, instanceDir=collection3, schema=schema.xml, loadOnStartup=false, configSetProperties=configsetprops.json, name=collectionLazy3, transient=on, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-002/collection3/}
   [junit4]   2> 13171 INFO  (TEST-TestLazyCores.testLazySearch-seed#[604EA955F55DFDE5]) [    x:collectionLazy3] o.a.s.c.SolrCore Created CoreDescriptor: {instanceDir=collection4, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-002/collection4/, dataDir=data/, loadOnStartup=false, schema=schema.xml, name=collectionLazy4, config=solrconfig.xml, configSetProperties=configsetprops.json, transient=false}
   [junit4]   2> 13172 INFO  (TEST-TestLazyCores.testLazySearch-seed#[604EA955F55DFDE5]) [    x:collectionLazy3] o.a.s.c.SolrCore Created CoreDescriptor: {name=collectionLazy5, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-002/collection5/, config=solrconfig.xml, loadOnStartup=true, schema=schema.xml, instanceDir=collection5, configSetProperties=configsetprops.json, dataDir=data/, transient=false}
   [junit4]   2> 13172 INFO  (TEST-TestLazyCores.testLazySearch-seed#[604EA955F55DFDE5]) [    x:collectionLazy3] o.a.s.c.SolrCore Created CoreDescriptor: {loadOnStartup=false, config=solrconfig.xml, dataDir=data/, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-002/collection6/, name=collectionLazy6, transient=true, schema=schema.xml, configSetProperties=configsetprops.json, instanceDir=collection6}
   [junit4]   2> 13172 INFO  (TEST-TestLazyCores.testLazySearch-seed#[604EA955F55DFDE5]) [    x:collectionLazy3] o.a.s.c.SolrCore Created CoreDescriptor: {configSetProperties=configsetprops.json, name=collectionLazy7, transient=true, schema=schema.xml, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-002/collection7/, config=solrconfig.xml, instanceDir=collection7, dataDir=data/, loadOnStartup=false}
   [junit4]   2> 13173 INFO  (TEST-TestLazyCores.testLazySearch-seed#[604EA955F55DFDE5]) [    x:collectionLazy3] o.a.s.c.SolrCore Created CoreDescriptor: {configSetProperties=configsetprops.json, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-002/collection8/, config=solrconfig.xml, loadOnStartup=false, name=collectionLazy8, schema=schema.xml, dataDir=data/, transient=true, instanceDir=collection8}
   [junit4]   2> 13173 INFO  (TEST-TestLazyCores.testLazySearch-seed#[604EA955F55DFDE5]) [    x:collectionLazy3] o.a.s.c.SolrCore Created CoreDescriptor: {name=collectionLazy9, schema=schema.xml, configSetProperties=configsetprops.json, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-002/collection9/, instanceDir=collection9, loadOnStartup=false, dataDir=data/, config=solrconfig.xml, transient=true}
   [junit4]   2> 13174 INFO  (coreLoadExecutor-49-thread-1-processing-x:collectionLazy3) [    x:collectionLazy3] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-002/collection1/'
   [junit4]   2> 13178 INFO  (coreLoadExecutor-49-thread-2-processing-x:collectionLazy3) [    x:collectionLazy3] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-002/collection2/'
   [junit4]   2> 13178 INFO  (coreLoadExecutor-49-thread-1-processing-x:collectionLazy3) [    x:collectionLazy3] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 13178 INFO  (coreLoadExecutor-49-thread-2-processing-x:collectionLazy3) [    x:collectionLazy3] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 13179 INFO  (coreLoadExecutor-49-thread-1-processing-x:collectionLazy3) [    x:collectionLazy3] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 13179 INFO  (coreLoadExecutor-49-thread-2-processing-x:collectionLazy3) [    x:collectionLazy3] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 13177 INFO  (coreLoadExecutor-49-thread-3-processing-x:collectionLazy3) [    x:collectionLazy3] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-002/collection5/'
   [junit4]   2> 13179 INFO  (coreLoadExecutor-49-thread-3-processing-x:collectionLazy3) [    x:collectionLazy3] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 13179 INFO  (coreLoadExecutor-49-thread-3-processing-x:collectionLazy3) [    x:collectionLazy3] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 13264 INFO  (coreLoadExecutor-49-thread-3-processing-x:collectionLazy3) [    x:collectionLazy3] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 13271 INFO  (coreLoadExecutor-49-thread-2-processing-x:collectionLazy3) [    x:collectionLazy3] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 13279 INFO  (coreLoadExecutor-49-thread-1-processing-x:collectionLazy3) [    x:collectionLazy3] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 13314 INFO  (coreLoadExecutor-49-thread-3-processing-x:collectionLazy3) [    x:collectionLazy3] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 13322 INFO  (coreLoadExecutor-49-thread-2-processing-x:collectionLazy3) [    x:collectionLazy3] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 13348 INFO  (coreLoadExecutor-49-thread-1-processing-x:collectionLazy3) [    x:collectionLazy3] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 13426 INFO  (coreLoadExecutor-49-thread-3-processing-x:collectionLazy3) [    x:collectionLazy3] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 13426 INFO  (coreLoadExecutor-49-thread-3-processing-x:collectionLazy3) [    x:collectionLazy3] o.a.s.s.IndexSchema Reading Solr Schema from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-002/collection5/conf/schema.xml
   [junit4]   2> 13449 INFO  (coreLoadExecutor-49-thread-3-processing-x:collectionLazy3) [    x:collectionLazy3] o.a.s.s.IndexSchema [collectionLazy5] Schema name=tiny
   [junit4]   2> 13452 INFO  (coreLoadExecutor-49-thread-2-processing-x:collectionLazy3) [    x:collectionLazy3] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 13452 INFO  (coreLoadExecutor-49-thread-2-processing-x:collectionLazy3) [    x:collectionLazy3] o.a.s.s.IndexSchema Reading Solr Schema from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-002/collection2/conf/schema.xml
   [junit4]   2> 13471 INFO  (coreLoadExecutor-49-thread-1-processing-x:collectionLazy3) [    x:collectionLazy3] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 13472 INFO  (coreLoadExecutor-49-thread-1-processing-x:collectionLazy3) [    x:collectionLazy3] o.a.s.s.IndexSchema Reading Solr Schema from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-002/collection1/conf/schema.xml
   [junit4]   2> 13475 INFO  (coreLoadExecutor-49-thread-2-processing-x:collectionLazy3) [    x:collectionLazy3] o.a.s.s.IndexSchema [collectionLazy2] Schema name=tiny
   [junit4]   2> 13478 INFO  (coreLoadExecutor-49-thread-3-processing-x:collectionLazy3) [    x:collectionLazy3] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 13480 INFO  (coreLoadExecutor-49-thread-3-processing-x:collectionLazy3) [    x:collectionLazy3] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-002/collection5/conf'
   [junit4]   2> 13480 INFO  (coreLoadExecutor-49-thread-3-processing-x:collectionLazy3) [    x:collectionLazy3] o.a.s.c.CoreContainer Creating SolrCore 'collectionLazy5' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-002/collection5/
   [junit4]   2> 13480 INFO  (coreLoadExecutor-49-thread-3-processing-x:collectionLazy3) [    x:collectionLazy5] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 13481 INFO  (coreLoadExecutor-49-thread-3-processing-x:collectionLazy3) [    x:collectionLazy5] o.a.s.c.SolrCore [[collectionLazy5] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_604EA955F55DFDE5-001/tempDir-002/collection5/], dataDir=[null]
   [junit4]   2> 13481 INFO  (coreLoadExecutor-49-thread-3-processing-x:collectionLazy3) [    x:collectionLazy5] o.a.s.c.SolrCore JMX monitoring not detected for core: collectionLazy5
   [junit4]   2> 13482 INFO  (coreLoadExecutor-49-thread-3-processing-x:collectionLazy3) [    x:collectionLazy5] o.a.s.c

[...truncated too long message...]

onExpiredException: KeeperErrorCode = Session expired for /collections/collection1/leader_elect/shard1/election/94741961935421458-core_node2-n_0000000003
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.delete(ZooKeeper.java:873)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$2.execute(SolrZkClient.java:244)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$2.execute(SolrZkClient.java:241)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:61)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.delete(SolrZkClient.java:241)
   [junit4]   2> 	at org.apache.solr.cloud.ElectionContext.cancelElection(ElectionContext.java:85)
   [junit4]   2> 	at org.apache.solr.cloud.ShardLeaderElectionContextBase.cancelElection(ElectionContext.java:166)
   [junit4]   2> 	at org.apache.solr.cloud.ShardLeaderElectionContext.cancelElection(ElectionContext.java:268)
   [junit4]   2> 	at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:450)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.access$200(LeaderElector.java:58)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector$ElectionWatcher.process(LeaderElector.java:388)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$3$1.run(SolrZkClient.java:264)
   [junit4]   2> 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
   [junit4]   2> 	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
   [junit4]   2> 	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 434742 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[604EA955F55DFDE5]) [    ] o.e.j.s.ServerConnector Stopped ServerConnector@1d923336{HTTP/1.1}{127.0.0.1:37038}
   [junit4]   2> 434743 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[604EA955F55DFDE5]) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@48bc20ba{/mcfnu/hs,null,UNAVAILABLE}
   [junit4]   2> 434745 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[604EA955F55DFDE5]) [n:127.0.0.1:37038_mcfnu%2Fhs c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.ZkTestServer connecting to 127.0.0.1:40546 40546
   [junit4]   2> 434907 INFO  (Thread-112) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:40546 40546
   [junit4]   2> 434910 WARN  (Thread-112) [    ] o.a.s.c.ZkTestServer Watch limit violations: 
   [junit4]   2> Maximum concurrent create/delete watches above limit:
   [junit4]   2> 
   [junit4]   2> 	7	/solr/aliases.json
   [junit4]   2> 	6	/solr/security.json
   [junit4]   2> 	5	/solr/configs/conf1
   [junit4]   2> 
   [junit4]   2> Maximum concurrent data watches above limit:
   [junit4]   2> 
   [junit4]   2> 	36	/solr/clusterstate.json
   [junit4]   2> 	3	/solr/overseer_elect/election/94741961935421444-127.0.0.1:45521_mcfnu%2Fhs-n_0000000000
   [junit4]   2> 
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2> 
   [junit4]   2> 	11	/solr/live_nodes
   [junit4]   2> 	8	/solr/collections
   [junit4]   2> 	3	/solr/overseer/collection-queue-work
   [junit4]   2> 	3	/solr/overseer/queue
   [junit4]   2> 
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=LeaderInitiatedRecoveryOnShardRestartTest -Dtests.method=testRestartWithAllInLIR -Dtests.seed=604EA955F55DFDE5 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=be -Dtests.timezone=America/Dawson_Creek -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR    375s J1 | LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR <<<
   [junit4]    > Throwable #1: java.lang.AssertionError: There are still nodes recoverying - waited for 330 seconds
   [junit4]    > 	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:172)
   [junit4]    > 	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:133)
   [junit4]    > 	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:128)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.waitForRecoveriesToFinish(AbstractFullDistribZkTestBase.java:830)
   [junit4]    > 	at org.apache.solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR(LeaderInitiatedRecoveryOnShardRestartTest.java:81)
   [junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:963)
   [junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:938)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)Throwable #2: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=484, name=coreZkRegister-215-thread-1, state=RUNNABLE, group=TGRP-LeaderInitiatedRecoveryOnShardRestartTest]
   [junit4]    > Caused by: java.lang.AssertionError
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([604EA955F55DFDE5]:0)
   [junit4]    > 	at org.apache.solr.cloud.ZkController.updateLeaderInitiatedRecoveryState(ZkController.java:2133)
   [junit4]    > 	at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:434)
   [junit4]    > 	at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)
   [junit4]    > 	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)
   [junit4]    > 	at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:346)
   [junit4]    > 	at org.apache.solr.cloud.ZkController.joinElection(ZkController.java:1113)
   [junit4]    > 	at org.apache.solr.cloud.ZkController.register(ZkController.java:926)
   [junit4]    > 	at org.apache.solr.cloud.ZkController.register(ZkController.java:881)
   [junit4]    > 	at org.apache.solr.core.ZkContainer$2.run(ZkContainer.java:183)
   [junit4]    > 	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)
   [junit4]    > 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   [junit4]    > 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 434943 INFO  (SUITE-LeaderInitiatedRecoveryOnShardRestartTest-seed#[604EA955F55DFDE5]-worker) [n:127.0.0.1:37038_mcfnu%2Fhs c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_604EA955F55DFDE5-001
   [junit4]   2> Oct 23, 2015 8:53:07 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene53): {}, docValues:{}, sim=RandomSimilarityProvider(queryNorm=true,coord=crazy): {}, locale=be, timezone=America/Dawson_Creek
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_72 (64-bit)/cpus=4,threads=1,free=103223288,total=343932928
   [junit4]   2> NOTE: All tests run in this JVM: [DistanceFunctionTest, DocValuesTest, TestReplicaProperties, TestNRTOpen, TestUpdate, TestBM25SimilarityFactory, PreAnalyzedFieldTest, LeaderInitiatedRecoveryOnShardRestartTest]
   [junit4] Completed [406/542] on J1 in 375.63s, 1 test, 1 error <<< FAILURES!

[...truncated 430 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 542 suites (7 ignored), 2143 tests, 4 suite-level errors, 1 error, 1449 ignored (7 assumptions) [seed: 604EA955F55DFDE5]

Total time: 165 minutes 29 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 994 - Still Failing

[Apache Jenkins Server <je...@builds.apache.org>]

Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/994/

2 tests failed.
FAILED:  org.apache.lucene.codecs.lucene42.TestLucene42TermVectorsFormat.testRamBytesUsed

Error Message:
Test abandoned because suite timeout was reached.

Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
	at __randomizedtesting.SeedInfo.seed([57A52995A507B6FB]:0)


FAILED:  junit.framework.TestSuite.org.apache.lucene.codecs.lucene42.TestLucene42TermVectorsFormat

Error Message:
Suite timeout exceeded (>= 7200000 msec).

Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
	at __randomizedtesting.SeedInfo.seed([57A52995A507B6FB]:0)




Build Log:
[...truncated 5140 lines...]
   [junit4] Suite: org.apache.lucene.codecs.lucene42.TestLucene42TermVectorsFormat
   [junit4]   2> X 23, 2015 10:37:57 AM com.carrotsearch.randomizedtesting.ThreadLeakControl$2 evaluate
   [junit4]   2> WARNING: Suite execution timed out: org.apache.lucene.codecs.lucene42.TestLucene42TermVectorsFormat
   [junit4]   2>    1) Thread[id=144, name=SUITE-TestLucene42TermVectorsFormat-seed#[57A52995A507B6FB], state=RUNNABLE, group=TGRP-TestLucene42TermVectorsFormat]
   [junit4]   2>         at java.lang.Thread.getStackTrace(Thread.java:1589)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$4.run(ThreadLeakControl.java:688)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$4.run(ThreadLeakControl.java:685)
   [junit4]   2>         at java.security.AccessController.doPrivileged(Native Method)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl.getStackTrace(ThreadLeakControl.java:685)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl.getThreadsWithTraces(ThreadLeakControl.java:701)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl.formatThreadStacksFull(ThreadLeakControl.java:681)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl.access$1000(ThreadLeakControl.java:64)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$2.evaluate(ThreadLeakControl.java:414)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:676)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner.access$200(RandomizedRunner.java:140)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$2.run(RandomizedRunner.java:587)
   [junit4]   2>    2) Thread[id=8, name=JUnit4-serializer-daemon, state=TIMED_WAITING, group=main]
   [junit4]   2>         at java.lang.Thread.sleep(Native Method)
   [junit4]   2>         at com.carrotsearch.ant.tasks.junit4.events.Serializer$1.run(Serializer.java:49)
   [junit4]   2>    3) Thread[id=145, name=TEST-TestLucene42TermVectorsFormat.testRamBytesUsed-seed#[57A52995A507B6FB], state=RUNNABLE, group=TGRP-TestLucene42TermVectorsFormat]
   [junit4]   2>         at org.apache.lucene.util.packed.BlockPackedReaderIterator.skip(BlockPackedReaderIterator.java:145)
   [junit4]   2>         at org.apache.lucene.codecs.lucene42.Lucene42TermVectorsReader.readPositions(Lucene42TermVectorsReader.java:584)
   [junit4]   2>         at org.apache.lucene.codecs.lucene42.Lucene42TermVectorsReader.get(Lucene42TermVectorsReader.java:398)
   [junit4]   2>         at org.apache.lucene.codecs.TermVectorsWriter.merge(TermVectorsWriter.java:194)
   [junit4]   2>         at org.apache.lucene.index.SegmentMerger.mergeVectors(SegmentMerger.java:187)
   [junit4]   2>         at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:127)
   [junit4]   2>         at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4068)
   [junit4]   2>         at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3648)
   [junit4]   2>         at org.apache.lucene.index.SerialMergeScheduler.merge(SerialMergeScheduler.java:40)
   [junit4]   2>         at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:1929)
   [junit4]   2>         at org.apache.lucene.index.IndexWriter.doAfterSegmentFlushed(IndexWriter.java:4712)
   [junit4]   2>         at org.apache.lucene.index.DocumentsWriter$MergePendingEvent.process(DocumentsWriter.java:695)
   [junit4]   2>         at org.apache.lucene.index.IndexWriter.processEvents(IndexWriter.java:4738)
   [junit4]   2>         at org.apache.lucene.index.IndexWriter.processEvents(IndexWriter.java:4729)
   [junit4]   2>         at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1476)
   [junit4]   2>         at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1254)
   [junit4]   2>         at org.apache.lucene.index.BaseIndexFileFormatTestCase.testRamBytesUsed(BaseIndexFileFormatTestCase.java:262)
   [junit4]   2>         at org.apache.lucene.index.BaseTermVectorsFormatTestCase.testRamBytesUsed(BaseTermVectorsFormatTestCase.java:74)
   [junit4]   2>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   2>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   [junit4]   2>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2>         at java.lang.reflect.Method.invoke(Method.java:606)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1665)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:864)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:900)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:914)
   [junit4]   2>         at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
   [junit4]   2>         at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2>         at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
   [junit4]   2>         at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2>         at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:873)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:775)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:809)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:820)
   [junit4]   2>         at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   2>         at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2>         at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2>         at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
   [junit4]   2>         at java.lang.Thread.run(Thread.java:745)
   [junit4]   2>    4) Thread[id=1, name=main, state=WAITING, group=main]
   [junit4]   2>         at java.lang.Object.wait(Native Method)
   [junit4]   2>         at java.lang.Thread.join(Thread.java:1281)
   [junit4]   2>         at java.lang.Thread.join(Thread.java:1355)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:597)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner.run(RandomizedRunner.java:446)
   [junit4]   2>         at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.execute(SlaveMain.java:244)
   [junit4]   2>         at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.main(SlaveMain.java:355)
   [junit4]   2>         at com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe.main(SlaveMainSafe.java:10)
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestLucene42TermVectorsFormat -Dtests.method=testRamBytesUsed -Dtests.seed=57A52995A507B6FB -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=bg_BG -Dtests.timezone=ACT -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
   [junit4] ERROR   7200s J0 | TestLucene42TermVectorsFormat.testRamBytesUsed <<<
   [junit4]    > Throwable #1: java.lang.Exception: Test abandoned because suite timeout was reached.
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([57A52995A507B6FB]:0)
   [junit4]   2> X 23, 2015 10:37:57 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
   [junit4]   2> X 23, 2015 10:38:17 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> SEVERE: 1 thread leaked from SUITE scope at org.apache.lucene.codecs.lucene42.TestLucene42TermVectorsFormat: 
   [junit4]   2>    1) Thread[id=145, name=TEST-TestLucene42TermVectorsFormat.testRamBytesUsed-seed#[57A52995A507B6FB], state=RUNNABLE, group=TGRP-TestLucene42TermVectorsFormat]
   [junit4]   2>         at org.apache.lucene.store.MockIndexInputWrapper.readByte(MockIndexInputWrapper.java:132)
   [junit4]   2>         at org.apache.lucene.util.packed.BlockPackedReaderIterator.skip(BlockPackedReaderIterator.java:126)
   [junit4]   2>         at org.apache.lucene.codecs.lucene42.Lucene42TermVectorsReader.readPositions(Lucene42TermVectorsReader.java:584)
   [junit4]   2>         at org.apache.lucene.codecs.lucene42.Lucene42TermVectorsReader.get(Lucene42TermVectorsReader.java:398)
   [junit4]   2>         at org.apache.lucene.codecs.TermVectorsWriter.merge(TermVectorsWriter.java:194)
   [junit4]   2>         at org.apache.lucene.index.SegmentMerger.mergeVectors(SegmentMerger.java:187)
   [junit4]   2>         at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:127)
   [junit4]   2>         at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4068)
   [junit4]   2>         at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3648)
   [junit4]   2>         at org.apache.lucene.index.SerialMergeScheduler.merge(SerialMergeScheduler.java:40)
   [junit4]   2>         at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:1929)
   [junit4]   2>         at org.apache.lucene.index.IndexWriter.doAfterSegmentFlushed(IndexWriter.java:4712)
   [junit4]   2>         at org.apache.lucene.index.DocumentsWriter$MergePendingEvent.process(DocumentsWriter.java:695)
   [junit4]   2>         at org.apache.lucene.index.IndexWriter.processEvents(IndexWriter.java:4738)
   [junit4]   2>         at org.apache.lucene.index.IndexWriter.processEvents(IndexWriter.java:4729)
   [junit4]   2>         at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1476)
   [junit4]   2>         at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1254)
   [junit4]   2>         at org.apache.lucene.index.BaseIndexFileFormatTestCase.testRamBytesUsed(BaseIndexFileFormatTestCase.java:262)
   [junit4]   2>         at org.apache.lucene.index.BaseTermVectorsFormatTestCase.testRamBytesUsed(BaseTermVectorsFormatTestCase.java:74)
   [junit4]   2>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   2>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   [junit4]   2>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2>         at java.lang.reflect.Method.invoke(Method.java:606)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1665)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:864)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:900)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:914)
   [junit4]   2>         at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
   [junit4]   2>         at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2>         at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
   [junit4]   2>         at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2>         at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:873)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:775)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:809)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:820)
   [junit4]   2>         at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   2>         at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2>         at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2>         at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
   [junit4]   2>         at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> X 23, 2015 10:38:17 AM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
   [junit4]   2> INFO: Starting to interrupt leaked threads:
   [junit4]   2>    1) Thread[id=145, name=TEST-TestLucene42TermVectorsFormat.testRamBytesUsed-seed#[57A52995A507B6FB], state=RUNNABLE, group=TGRP-TestLucene42TermVectorsFormat]
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/backward-codecs/test/J0/temp/lucene.codecs.lucene42.TestLucene42TermVectorsFormat_57A52995A507B6FB-001
   [junit4]   2> Oct 22, 2015 9:08:19 PM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
   [junit4]   2> INFO: All leaked threads terminated.
   [junit4]   2> NOTE: test params are: codec=Lucene53, sim=DefaultSimilarity, locale=bg_BG, timezone=ACT
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_72 (64-bit)/cpus=4,threads=1,free=210661928,total=511180800
   [junit4]   2> NOTE: All tests run in this JVM: [TestLucene40TermVectorsFormat, TestLucene46UndeadNorms, TestLucene41PostingsFormat2, TestLucene46FieldInfoFormat, TestReuseDocsEnum, TestLucene42TermVectorsFormat]
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestLucene42TermVectorsFormat -Dtests.seed=57A52995A507B6FB -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=bg_BG -Dtests.timezone=ACT -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
   [junit4] ERROR   0.00s J0 | TestLucene42TermVectorsFormat (suite) <<<
   [junit4]    > Throwable #1: java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([57A52995A507B6FB]:0)
   [junit4] Completed [30/30] on J0 in 7221.92s, 6 tests, 2 errors <<< FAILURES!

[...truncated 12 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build.xml:471: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:2248: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/module-build.xml:58: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 30 suites, 789 tests, 1 suite-level error, 1 error, 81 ignored (81 assumptions) [seed: 57A52995A507B6FB]

Total time: 228 minutes 34 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

Re: [JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 993 - Still Failing

[Shalin Shekhar Mangar <sh...@gmail.com>]

This is a valid bug caused by assertions added in SOLR-8069. See
https://issues.apache.org/jira/browse/SOLR-8069?focusedCommentId=14968664&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14968664

On Thu, Oct 22, 2015 at 12:17 PM, Apache Jenkins Server
<je...@builds.apache.org> wrote:
> Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/993/
>
> 1 tests failed.
> FAILED:  org.apache.solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR
>
> Error Message:
> Captured an uncaught exception in thread: Thread[id=43491, name=coreZkRegister-5997-thread-1, state=RUNNABLE, group=TGRP-LeaderInitiatedRecoveryOnShardRestartTest]
>
> Stack Trace:
> com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=43491, name=coreZkRegister-5997-thread-1, state=RUNNABLE, group=TGRP-LeaderInitiatedRecoveryOnShardRestartTest]
> Caused by: java.lang.AssertionError
>         at __randomizedtesting.SeedInfo.seed([7F78F76DDF75FAD1]:0)
>         at org.apache.solr.cloud.ZkController.updateLeaderInitiatedRecoveryState(ZkController.java:2133)
>         at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:434)
>         at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)
>         at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)
>         at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:346)
>         at org.apache.solr.cloud.ZkController.joinElection(ZkController.java:1113)
>         at org.apache.solr.cloud.ZkController.register(ZkController.java:926)
>         at org.apache.solr.cloud.ZkController.register(ZkController.java:881)
>         at org.apache.solr.core.ZkContainer$2.run(ZkContainer.java:183)
>         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>         at java.lang.Thread.run(Thread.java:745)
>
>
>
>
> Build Log:
> [...truncated 11227 lines...]
>    [junit4] Suite: org.apache.solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest
>    [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/init-core-data-001
>    [junit4]   2> 2804605 INFO  (SUITE-LeaderInitiatedRecoveryOnShardRestartTest-seed#[7F78F76DDF75FAD1]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
>    [junit4]   2> 2804608 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
>    [junit4]   2> 2804609 INFO  (Thread-33860) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
>    [junit4]   2> 2804609 INFO  (Thread-33860) [    ] o.a.s.c.ZkTestServer Starting server
>    [junit4]   2> 2804709 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.ZkTestServer start zk server on port:52462
>    [junit4]   2> 2804709 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
>    [junit4]   2> 2804709 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
>    [junit4]   2> 2804712 INFO  (zkCallback-2280-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7a70d2b6 name:ZooKeeperConnection Watcher:127.0.0.1:52462 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
>    [junit4]   2> 2804712 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
>    [junit4]   2> 2804712 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
>    [junit4]   2> 2804712 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
>    [junit4]   2> 2804715 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
>    [junit4]   2> 2804715 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
>    [junit4]   2> 2804716 INFO  (zkCallback-2281-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5128e657 name:ZooKeeperConnection Watcher:127.0.0.1:52462/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
>    [junit4]   2> 2804716 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
>    [junit4]   2> 2804716 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
>    [junit4]   2> 2804716 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
>    [junit4]   2> 2804718 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
>    [junit4]   2> 2804719 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
>    [junit4]   2> 2804720 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
>    [junit4]   2> 2804722 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
>    [junit4]   2> 2804722 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
>    [junit4]   2> 2804724 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
>    [junit4]   2> 2804724 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
>    [junit4]   2> 2804726 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
>    [junit4]   2> 2804726 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
>    [junit4]   2> 2804728 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
>    [junit4]   2> 2804728 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
>    [junit4]   2> 2804729 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
>    [junit4]   2> 2804730 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
>    [junit4]   2> 2804731 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
>    [junit4]   2> 2804732 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
>    [junit4]   2> 2804733 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
>    [junit4]   2> 2804733 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
>    [junit4]   2> 2804738 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
>    [junit4]   2> 2804738 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
>    [junit4]   2> 2804739 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
>    [junit4]   2> 2804740 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
>    [junit4]   2> 2804741 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
>    [junit4]   2> 2804741 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
>    [junit4]   2> 2804743 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
>    [junit4]   2> 2804743 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
>    [junit4]   2> 2805008 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/control-001/cores/collection1
>    [junit4]   2> 2805010 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
>    [junit4]   2> 2805012 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@665b505c{/,null,AVAILABLE}
>    [junit4]   2> 2805013 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.e.j.s.ServerConnector Started ServerConnector@68a3874{HTTP/1.1}{127.0.0.1:56746}
>    [junit4]   2> 2805013 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.e.j.s.Server Started @2807895ms
>    [junit4]   2> 2805013 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostPort=56746, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/tempDir-001/control/data, hostContext=/, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/control-001/cores}
>    [junit4]   2> 2805013 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@1fe91485
>    [junit4]   2> 2805013 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/control-001/'
>    [junit4]   2> 2805013 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
>    [junit4]   2> 2805013 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
>    [junit4]   2> 2805027 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
>    [junit4]   2> 2805028 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
>    [junit4]   2> 2805029 INFO  (zkCallback-2282-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@39baa32 name:ZooKeeperConnection Watcher:127.0.0.1:52462/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
>    [junit4]   2> 2805029 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
>    [junit4]   2> 2805029 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
>    [junit4]   2> 2805031 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
>    [junit4]   2> 2805031 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/control-001/solr.xml
>    [junit4]   2> 2805043 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/control-001/cores
>    [junit4]   2> 2805043 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.CoreContainer New CoreContainer 1984045032
>    [junit4]   2> 2805043 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/control-001/]
>    [junit4]   2> 2805044 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/control-001/lib
>    [junit4]   2> 2805044 WARN  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/control-001/lib).
>    [junit4]   2> 2805051 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
>    [junit4]   2> 2805052 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
>    [junit4]   2> 2805052 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
>    [junit4]   2> 2805052 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
>    [junit4]   2> 2805052 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:52462/solr
>    [junit4]   2> 2805052 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.ZkController zkHost includes chroot
>    [junit4]   2> 2805052 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
>    [junit4]   2> 2805054 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
>    [junit4]   2> 2805054 INFO  (zkCallback-2284-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2e9d74e9 name:ZooKeeperConnection Watcher:127.0.0.1:52462 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
>    [junit4]   2> 2805054 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
>    [junit4]   2> 2805055 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
>    [junit4]   2> 2805056 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
>    [junit4]   2> 2805057 INFO  (zkCallback-2285-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@51ed1947 name:ZooKeeperConnection Watcher:127.0.0.1:52462/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
>    [junit4]   2> 2805058 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
>    [junit4]   2> 2805058 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
>    [junit4]   2> 2805061 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
>    [junit4]   2> 2805063 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
>    [junit4]   2> 2805065 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
>    [junit4]   2> 2805066 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
>    [junit4]   2> 2805068 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
>    [junit4]   2> 2805069 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
>    [junit4]   2> 2805070 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
>    [junit4]   2> 2805071 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.c.c.SolrZkClient makePath: /security.json
>    [junit4]   2> 2805072 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:56746_
>    [junit4]   2> 2805073 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:56746_
>    [junit4]   2> 2805074 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
>    [junit4]   2> 2805075 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
>    [junit4]   2> 2805076 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.c.Overseer Overseer (id=null) closing
>    [junit4]   2> 2805077 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94731933327425540-127.0.0.1:56746_-n_0000000000
>    [junit4]   2> 2805077 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:56746_
>    [junit4]   2> 2805077 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
>    [junit4]   2> 2805078 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.c.Overseer Overseer (id=94731933327425540-127.0.0.1:56746_-n_0000000000) starting
>    [junit4]   2> 2805080 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
>    [junit4]   2> 2805085 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
>    [junit4]   2> 2805085 INFO  (OverseerCollectionConfigSetProcessor-94731933327425540-127.0.0.1:56746_-n_0000000000) [n:127.0.0.1:56746_    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
>    [junit4]   2> 2805085 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper...
>    [junit4]   2> 2805085 INFO  (OverseerStateUpdate-94731933327425540-127.0.0.1:56746_-n_0000000000) [n:127.0.0.1:56746_    ] o.a.s.c.Overseer Starting to work on the main queue
>    [junit4]   2> 2805108 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
>    [junit4]   2> 2805109 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.c.CoreContainer No authentication plugin used.
>    [junit4]   2> 2805110 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/control-001/cores
>    [junit4]   2> 2805110 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.c.SolrCore Created CoreDescriptor: {loadOnStartup=true, configSetProperties=configsetprops.json, coreNodeName=, dataDir=data/, name=collection1, config=solrconfig.xml, collection=control_collection, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/control-001/cores/collection1/, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/control-001/cores/collection1, shard=, transient=false, schema=schema.xml}
>    [junit4]   2> 2805110 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/control-001/cores/collection1/
>    [junit4]   2> 2805111 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.c.CoresLocator Found 1 core definitions
>    [junit4]   2> 2805111 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
>    [junit4]   2> 2805111 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
>    [junit4]   2> 2805112 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
>    [junit4]   2> 2805112 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
>    [junit4]   2> 2805113 INFO  (OverseerStateUpdate-94731933327425540-127.0.0.1:56746_-n_0000000000) [n:127.0.0.1:56746_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
>    [junit4]   2>   "operation":"state",
>    [junit4]   2>   "shard":null,
>    [junit4]   2>   "roles":null,
>    [junit4]   2>   "base_url":"http://127.0.0.1:56746",
>    [junit4]   2>   "state":"down",
>    [junit4]   2>   "collection":"control_collection",
>    [junit4]   2>   "core":"collection1",
>    [junit4]   2>   "numShards":"1",
>    [junit4]   2>   "node_name":"127.0.0.1:56746_"} current state version: 0
>    [junit4]   2> 2805113 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
>    [junit4]   2> 2805114 INFO  (OverseerStateUpdate-94731933327425540-127.0.0.1:56746_-n_0000000000) [n:127.0.0.1:56746_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
>    [junit4]   2>   "operation":"state",
>    [junit4]   2>   "shard":null,
>    [junit4]   2>   "roles":null,
>    [junit4]   2>   "base_url":"http://127.0.0.1:56746",
>    [junit4]   2>   "state":"down",
>    [junit4]   2>   "collection":"control_collection",
>    [junit4]   2>   "core":"collection1",
>    [junit4]   2>   "numShards":"1",
>    [junit4]   2>   "node_name":"127.0.0.1:56746_"}
>    [junit4]   2> 2805114 INFO  (OverseerStateUpdate-94731933327425540-127.0.0.1:56746_-n_0000000000) [n:127.0.0.1:56746_    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
>    [junit4]   2> 2805114 INFO  (OverseerStateUpdate-94731933327425540-127.0.0.1:56746_-n_0000000000) [n:127.0.0.1:56746_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
>    [junit4]   2> 2805116 INFO  (zkCallback-2285-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
>    [junit4]   2> 2806114 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
>    [junit4]   2> 2806114 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
>    [junit4]   2> 2806115 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
>    [junit4]   2> 2806115 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
>    [junit4]   2> 2806115 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
>    [junit4]   2> 2806116 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/control-001/cores/collection1/'
>    [junit4]   2> 2806116 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
>    [junit4]   2> 2806116 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
>    [junit4]   2> 2806129 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0
>    [junit4]   2> 2806133 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
>    [junit4]   2> 2806145 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
>    [junit4]   2> 2806170 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
>    [junit4]   2> 2806171 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
>    [junit4]   2> 2806177 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
>    [junit4]   2> 2806414 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
>    [junit4]   2> 2806439 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
>    [junit4]   2> 2806440 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
>    [junit4]   2> 2806448 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
>    [junit4]   2> 2806451 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
>    [junit4]   2> 2806454 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
>    [junit4]   2> 2806454 WARN  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
>    [junit4]   2> 2806454 WARN  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
>    [junit4]   2> 2806454 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
>    [junit4]   2> 2806455 WARN  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
>    [junit4]   2> 2806455 WARN  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
>    [junit4]   2> 2806456 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
>    [junit4]   2> 2806456 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
>    [junit4]   2> 2806456 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
>    [junit4]   2> 2806456 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/control-001/cores/collection1/], dataDir=[null]
>    [junit4]   2> 2806456 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@379f19b4
>    [junit4]   2> 2806457 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/control-001/cores/collection1/data
>    [junit4]   2> 2806457 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/control-001/cores/collection1/data/index/
>    [junit4]   2> 2806457 WARN  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
>    [junit4]   2> 2806457 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/control-001/cores/collection1/data/index
>    [junit4]   2> 2806458 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@7cd8dcec
>    [junit4]   2> 2806458 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
>    [junit4]   2>        commit{dir=MockDirectoryWrapper(RAMDirectory@19049fd lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@27e981d8),segFN=segments_1,generation=1}
>    [junit4]   2> 2806458 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
>    [junit4]   2> 2806458 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/control-001/cores/collection1/data/
>    [junit4]   2> 2806459 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
>    [junit4]   2> 2806462 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
>    [junit4]   2> 2806462 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
>    [junit4]   2> 2806462 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
>    [junit4]   2> 2806462 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
>    [junit4]   2> 2806462 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
>    [junit4]   2> 2806462 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
>    [junit4]   2> 2806462 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
>    [junit4]   2> 2806462 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
>    [junit4]   2> 2806463 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
>    [junit4]   2> 2806465 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
>    [junit4]   2> 2806466 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
>    [junit4]   2> 2806468 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
>    [junit4]   2> 2806469 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
>    [junit4]   2> 2806472 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/logging,/update/json,/admin/segments,/admin/mbeans,/admin/ping,/update/csv,/admin/plugins,/update,/admin/file,/replication,standard,/update/json/docs,/admin/threads,/admin/properties,/admin/luke,/admin/system,/get,/schema,/config
>    [junit4]   2> 2806473 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
>    [junit4]   2> 2806475 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
>    [junit4]   2> 2806475 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
>    [junit4]   2> 2806475 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
>    [junit4]   2> 2806476 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
>    [junit4]   2> 2806476 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=12, maxMergeAtOnceExplicit=36, maxMergedSegmentMB=82.4736328125, floorSegmentMB=1.5283203125, forceMergeDeletesPctAllowed=4.112648727330555, segmentsPerTier=14.0, maxCFSSegmentSizeMB=1.6865234375, noCFSRatio=0.0
>    [junit4]   2> 2806476 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
>    [junit4]   2>        commit{dir=MockDirectoryWrapper(RAMDirectory@19049fd lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@27e981d8),segFN=segments_1,generation=1}
>    [junit4]   2> 2806476 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
>    [junit4]   2> 2806476 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@772028e6[collection1] main
>    [junit4]   2> 2806477 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
>    [junit4]   2> 2806478 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
>    [junit4]   2> 2806478 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
>    [junit4]   2> 2806478 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
>    [junit4]   2> 2806478 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
>    [junit4]   2> 2806478 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
>    [junit4]   2> 2806479 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
>    [junit4]   2> 2806479 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
>    [junit4]   2> 2806479 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
>    [junit4]   2> 2806479 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
>    [junit4]   2> 2806479 INFO  (searcherExecutor-5949-thread-1-processing-r:core_node1 s:shard1 x:collection1 c:control_collection n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@772028e6[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
>    [junit4]   2> 2806479 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
>    [junit4]   2> 2806479 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
>    [junit4]   2> 2806479 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
>    [junit4]   2> 2806480 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1515710935199645696
>    [junit4]   2> 2806482 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 2.0ms to seed version buckets with highest version 1515710935199645696
>    [junit4]   2> 2806482 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
>    [junit4]   2> 2806482 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
>    [junit4]   2> 2806482 INFO  (coreZkRegister-5942-thread-1-processing-r:core_node1 s:shard1 x:collection1 c:control_collection n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:56746 collection:control_collection shard:shard1
>    [junit4]   2> 2806483 INFO  (coreZkRegister-5942-thread-1-processing-r:core_node1 s:shard1 x:collection1 c:control_collection n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
>    [junit4]   2> 2806486 INFO  (coreZkRegister-5942-thread-1-processing-r:core_node1 s:shard1 x:collection1 c:control_collection n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94731933327425540-core_node1-n_0000000000
>    [junit4]   2> 2806487 INFO  (coreZkRegister-5942-thread-1-processing-r:core_node1 s:shard1 x:collection1 c:control_collection n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
>    [junit4]   2> 2806488 INFO  (coreZkRegister-5942-thread-1-processing-r:core_node1 s:shard1 x:collection1 c:control_collection n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
>    [junit4]   2> 2806488 INFO  (coreZkRegister-5942-thread-1-processing-r:core_node1 s:shard1 x:collection1 c:control_collection n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
>    [junit4]   2> 2806488 INFO  (coreZkRegister-5942-thread-1-processing-r:core_node1 s:shard1 x:collection1 c:control_collection n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:56746/collection1/
>    [junit4]   2> 2806488 INFO  (coreZkRegister-5942-thread-1-processing-r:core_node1 s:shard1 x:collection1 c:control_collection n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
>    [junit4]   2> 2806488 INFO  (OverseerStateUpdate-94731933327425540-127.0.0.1:56746_-n_0000000000) [n:127.0.0.1:56746_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
>    [junit4]   2>   "operation":"leader",
>    [junit4]   2>   "shard":"shard1",
>    [junit4]   2>   "collection":"control_collection"} current state version: 1
>    [junit4]   2> 2806488 INFO  (coreZkRegister-5942-thread-1-processing-r:core_node1 s:shard1 x:collection1 c:control_collection n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:56746/collection1/ has no replicas
>    [junit4]   2> 2806488 INFO  (coreZkRegister-5942-thread-1-processing-r:core_node1 s:shard1 x:collection1 c:control_collection n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:56746/collection1/ shard1
>    [junit4]   2> 2806489 INFO  (coreZkRegister-5942-thread-1-processing-r:core_node1 s:shard1 x:collection1 c:control_collection n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
>    [junit4]   2> 2806491 INFO  (coreZkRegister-5942-thread-1-processing-r:core_node1 s:shard1 x:collection1 c:control_collection n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
>    [junit4]   2> 2806493 INFO  (OverseerStateUpdate-94731933327425540-127.0.0.1:56746_-n_0000000000) [n:127.0.0.1:56746_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
>    [junit4]   2>   "operation":"leader",
>    [junit4]   2>   "shard":"shard1",
>    [junit4]   2>   "collection":"control_collection",
>    [junit4]   2>   "base_url":"http://127.0.0.1:56746",
>    [junit4]   2>   "core":"collection1",
>    [junit4]   2>   "state":"active"} current state version: 1
>    [junit4]   2> 2806595 INFO  (zkCallback-2285-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
>    [junit4]   2> 2806613 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
>    [junit4]   2> 2806614 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
>    [junit4]   2> 2806615 INFO  (zkCallback-2287-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@794415fc name:ZooKeeperConnection Watcher:127.0.0.1:52462/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
>    [junit4]   2> 2806615 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
>    [junit4]   2> 2806615 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
>    [junit4]   2> 2806615 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper...
>    [junit4]   2> 2806617 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
>    [junit4]   2> 2806617 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with stateFormat=2
>    [junit4]   2> 2806617 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
>    [junit4]   2> 2806618 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
>    [junit4]   2> 2806619 INFO  (zkCallback-2288-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@77c64dbb name:ZooKeeperConnection Watcher:127.0.0.1:52462/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
>    [junit4]   2> 2806619 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
>    [junit4]   2> 2806619 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
>    [junit4]   2> 2806622 INFO  (OverseerStateUpdate-94731933327425540-127.0.0.1:56746_-n_0000000000) [n:127.0.0.1:56746_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
>    [junit4]   2>   "operation":"create",
>    [junit4]   2>   "name":"collection1",
>    [junit4]   2>   "numShards":"1",
>    [junit4]   2>   "stateFormat":"2"} current state version: 2
>    [junit4]   2> 2806622 INFO  (OverseerStateUpdate-94731933327425540-127.0.0.1:56746_-n_0000000000) [n:127.0.0.1:56746_    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
>    [junit4]   2> 2806644 INFO  (coreZkRegister-5942-thread-1-processing-r:core_node1 s:shard1 x:collection1 c:control_collection n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:56746/collection1/ and leader is http://127.0.0.1:56746/collection1/
>    [junit4]   2> 2806644 INFO  (coreZkRegister-5942-thread-1-processing-r:core_node1 s:shard1 x:collection1 c:control_collection n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:56746
>    [junit4]   2> 2806644 INFO  (coreZkRegister-5942-thread-1-processing-r:core_node1 s:shard1 x:collection1 c:control_collection n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
>    [junit4]   2> 2806644 INFO  (coreZkRegister-5942-thread-1-processing-r:core_node1 s:shard1 x:collection1 c:control_collection n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
>    [junit4]   2> 2806644 INFO  (coreZkRegister-5942-thread-1-processing-r:core_node1 s:shard1 x:collection1 c:control_collection n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
>    [junit4]   2> 2806646 INFO  (OverseerStateUpdate-94731933327425540-127.0.0.1:56746_-n_0000000000) [n:127.0.0.1:56746_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
>    [junit4]   2>   "operation":"state",
>    [junit4]   2>   "shard":"shard1",
>    [junit4]   2>   "core_node_name":"core_node1",
>    [junit4]   2>   "roles":null,
>    [junit4]   2>   "base_url":"http://127.0.0.1:56746",
>    [junit4]   2>   "state":"active",
>    [junit4]   2>   "collection":"control_collection",
>    [junit4]   2>   "core":"collection1",
>    [junit4]   2>   "numShards":"1",
>    [junit4]   2>   "node_name":"127.0.0.1:56746_"} current state version: 2
>    [junit4]   2> 2806646 INFO  (OverseerStateUpdate-94731933327425540-127.0.0.1:56746_-n_0000000000) [n:127.0.0.1:56746_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
>    [junit4]   2>   "operation":"state",
>    [junit4]   2>   "shard":"shard1",
>    [junit4]   2>   "core_node_name":"core_node1",
>    [junit4]   2>   "roles":null,
>    [junit4]   2>   "base_url":"http://127.0.0.1:56746",
>    [junit4]   2>   "state":"active",
>    [junit4]   2>   "collection":"control_collection",
>    [junit4]   2>   "core":"collection1",
>    [junit4]   2>   "numShards":"1",
>    [junit4]   2>   "node_name":"127.0.0.1:56746_"}
>    [junit4]   2> 2806647 INFO  (OverseerStateUpdate-94731933327425540-127.0.0.1:56746_-n_0000000000) [n:127.0.0.1:56746_    ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/collection1/state.json
>    [junit4]   2> 2806750 INFO  (zkCallback-2287-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
>    [junit4]   2> 2806750 INFO  (zkCallback-2285-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
>    [junit4]   2> 2806891 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/shard-1-001/cores/collection1
>    [junit4]   2> 2806891 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/shard-1-001
>    [junit4]   2> 2806892 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
>    [junit4]   2> 2806894 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@719f3138{/,null,AVAILABLE}
>    [junit4]   2> 2806894 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.e.j.s.ServerConnector Started ServerConnector@3bc61657{HTTP/1.1}{127.0.0.1:50535}
>    [junit4]   2> 2806894 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.e.j.s.Server Started @2809776ms
>    [junit4]   2> 2806894 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/shard-1-001/cores, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/tempDir-001/jetty1, hostContext=/, solrconfig=solrconfig.xml, hostPort=50535}
>    [junit4]   2> 2806895 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@1fe91485
>    [junit4]   2> 2806895 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/shard-1-001/'
>    [junit4]   2> 2806895 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
>    [junit4]   2> 2806895 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
>    [junit4]   2> 2806909 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
>    [junit4]   2> 2806910 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
>    [junit4]   2> 2806911 INFO  (zkCallback-2289-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@306792d6 name:ZooKeeperConnection Watcher:127.0.0.1:52462/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
>    [junit4]   2> 2806911 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
>    [junit4]   2> 2806911 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
>    [junit4]   2> 2806913 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
>    [junit4]   2> 2806913 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/shard-1-001/solr.xml
>    [junit4]   2> 2806926 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/shard-1-001/cores
>    [junit4]   2> 2806926 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.CoreContainer New CoreContainer 1559122149
>    [junit4]   2> 2806926 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/shard-1-001/]
>    [junit4]   2> 2806926 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/shard-1-001/lib
>    [junit4]   2> 2806926 WARN  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/shard-1-001/lib).
>    [junit4]   2> 2806934 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
>    [junit4]   2> 2806935 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
>    [junit4]   2> 2806935 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
>    [junit4]   2> 2806935 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
>    [junit4]   2> 2806935 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:52462/solr
>    [junit4]   2> 2806935 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.ZkController zkHost includes chroot
>    [junit4]   2> 2806936 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
>    [junit4]   2> 2806936 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
>    [junit4]   2> 2806937 INFO  (zkCallback-2291-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@4c57b1e6 name:ZooKeeperConnection Watcher:127.0.0.1:52462 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
>    [junit4]   2> 2806937 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
>    [junit4]   2> 2806938 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
>    [junit4]   2> 2806940 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:50535_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
>    [junit4]   2> 2806941 INFO  (zkCallback-2292-thread-1-processing-n:127.0.0.1:50535_) [n:127.0.0.1:50535_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@124679be name:ZooKeeperConnection Watcher:127.0.0.1:52462/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
>    [junit4]   2> 2806941 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:50535_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
>    [junit4]   2> 2806945 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:50535_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper...
>    [junit4]   2> 2807951 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:50535_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50535_
>    [junit4]   2> 2807951 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:50535_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:50535_
>    [junit4]   2> 2807952 INFO  (zkCallback-2287-thread-1) [    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
>    [junit4]   2> 2807952 INFO  (zkCallback-2285-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
>    [junit4]   2> 2807952 INFO  (zkCallback-2292-thread-1-processing-n:127.0.0.1:50535_) [n:127.0.0.1:50535_    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
>    [junit4]   2> 2807954 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:50535_    ] o.a.s.c.Overseer Overseer (id=null) closing
>    [junit4]   2> 2807955 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:50535_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94731933327425545-127.0.0.1:50535_-n_0000000001
>    [junit4]   2> 2807955 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:50535_    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/94731933327425540-127.0.0.1:56746_-n_0000000000 to know if I could be the leader
>    [junit4]   2> 2807971 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:50535_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
>    [junit4]   2> 2807971 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:50535_    ] o.a.s.c.CoreContainer No authentication plugin used.
>    [junit4]   2> 2807972 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:50535_    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/shard-1-001/cores
>    [junit4]   2> 2807972 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:50535_    ] o.a.s.c.SolrCore Created CoreDescriptor: {collection=collection1, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/shard-1-001/cores/collection1/, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/shard-1-001/cores/collection1, name=collection1, configSetProperties=configsetprops.json, shard=, coreNodeName=, loadOnStartup=true, dataDir=data/, schema=schema.xml, transient=false, config=solrconfig.xml}
>    [junit4]   2> 2807973 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:50535_    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/shard-1-001/cores/collection1/
>    [junit4]   2> 2807973 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:50535_    ] o.a.s.c.CoresLocator Found 1 core definitions
>    [junit4]   2> 2807974 INFO  (coreLoadExecutor-5959-thread-1-processing-n:127.0.0.1:50535_) [n:127.0.0.1:50535_ c:collection1   x:collection1] o.a.s.c.ZkController publishing state=down
>    [junit4]   2> 2807974 INFO  (coreLoadExecutor-5959-thread-1-processing-n:127.0.0.1:50535_) [n:127.0.0.1:50535_ c:collection1   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
>    [junit4]   2> 2807974 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:50535_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
>    [junit4]   2> 2807974 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:50535_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
>    [junit4]   2> 2807975 INFO  (coreLoadExecutor-5959-thread-1-processing-n:127.0.0.1:50535_) [n:127.0.0.1:50535_ c:collection1   x:collection1] o.a.s.c.ZkController Registering watch for collection collection1
>    [junit4]   2> 2807975 INFO  (coreLoadExecutor-5959-thread-1-processing-n:127.0.0.1:50535_) [n:127.0.0.1:50535_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader addZkWatch collection1
>    [junit4]   2> 2807976 INFO  (OverseerStateUpdate-94731933327425540-127.0.0.1:56746_-n_0000000000) [n:127.0.0.1:56746_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
>    [junit4]   2>   "operation":"state",
>    [junit4]   2>   "shard":null,
>    [junit4]   2>   "roles":null,
>    [junit4]   2>   "base_url":"http://127.0.0.1:50535",
>    [junit4]   2>   "state":"down",
>    [junit4]   2>   "collection":"collection1",
>    [junit4]   2>   "core":"collection1",
>    [junit4]   2>   "numShards":"1",
>    [junit4]   2>   "node_name":"127.0.0.1:50535_"} current state version: 3
>    [junit4]   2> 2807976 INFO  (OverseerStateUpdate-94731933327425540-127.0.0.1:56746_-n_0000000000) [n:127.0.0.1:56746_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
>    [junit4]   2>   "operation":"state",
>    [junit4]   2>   "shard":null,
>    [junit4]   2>   "roles":null,
>    [junit4]   2>   "base_url":"http://127.0.0.1:50535",
>    [junit4]   2>   "state":"down",
>    [junit4]   2>   "collection":"collection1",
>    [junit4]   2>   "core":"collection1",
>    [junit4]   2>   "numShards":"1",
>    [junit4]   2>   "node_name":"127.0.0.1:50535_"}
>    [junit4]   2> 2807976 INFO  (OverseerStateUpdate-94731933327425540-127.0.0.1:56746_-n_0000000000) [n:127.0.0.1:56746_    ] o.a.s.c.o.ReplicaMutator Collection already exists with numShards=1
>    [junit4]   2> 2807976 INFO  (OverseerStateUpdate-94731933327425540-127.0.0.1:56746_-n_0000000000) [n:127.0.0.1:56746_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
>    [junit4]   2> 2807976 INFO  (coreLoadExecutor-5959-thread-1-processing-n:127.0.0.1:50535_) [n:127.0.0.1:50535_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Add data for collection1 ver 0
>    [junit4]   2> 2807977 INFO  (coreLoadExecutor-5959-thread-1-processing-n:127.0.0.1:50535_) [n:127.0.0.1:50535_ c:collection1   x:collection1] o.a.s.c.ZkController look for our core node name
>    [junit4]   2> 2808078 INFO  (OverseerStateUpdate-94731933327425540-127.0.0.1:56746_-n_0000000000) [n:127.0.0.1:56746_    ] o.a.s.c.o.ZkStateWriter going to update_collection /collections/collection1/state.json version: 0
>    [junit4]   2> 2808079 INFO  (zkCallback-2292-thread-1-processing-n:127.0.0.1:50535_) [n:127.0.0.1:50535_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json for collection collection1 has occurred - updating... (live nodes size: 2)
>    [junit4]   2> 2808079 INFO  (zkCallback-2292-thread-1-processing-n:127.0.0.1:50535_) [n:127.0.0.1:50535_    ] o.a.s.c.c.ZkStateReader Updating data for collection1 from 0 to 1
>    [junit4]   2> 2808977 INFO  (coreLoadExecutor-5959-thread-1-processing-n:127.0.0.1:50535_) [n:127.0.0.1:50535_ c:collection1   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
>    [junit4]   2> 2808977 INFO  (coreLoadExecutor-5959-thread-1-processing-n:127.0.0.1:50535_) [n:127.0.0.1:50535_ c:collection1   x:collection1] o.a.s.c.ZkController Check for collection zkNode:collection1
>    [junit4]   2> 2808978 INFO  (coreLoadExecutor-5959-thread-1-processing-n:127.0.0.1:50535_) [n:127.0.0.1:50535_ c:collection1   x:collection1] o.a.s.c.ZkController Collection zkNode exists
>    [junit4]   2> 2808978 INFO  (coreLoadExecutor-5959-thread-1-processing-n:127.0.0.1:50535_) [n:127.0.0.1:50535_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/collection1
>    [junit4]   2> 2808978 INFO  (coreLoadExecutor-5959-thread-1-processing-n:127.0.0.1:50535_) [n:127.0.0.1:50535_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
>    [junit4]   2> 2808978 INFO  (coreLoadExecutor-5959-thread-1-processing-n:127.0.0.1:50535_) [n:127.0.0.1:50535_ c:collection1   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/shard-1-001/cores/collection1/'
>    [junit4]   2> 2808979 INFO  (coreLoadExecutor-5959-thread-1-processing-n:127.0.0.1:50535_) [n:127.0.0.1:50535_ c:collection1   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
>    [junit4]   2> 2808979 INFO  (coreLoadExecutor-5959-thread-1-processing-n:127.0.0.1:50535_) [n:127.0.0.1:50535_ c:collection1   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
>    [junit4]   2> 2808992 INFO  (coreLoadExecutor-5959-thread-1-processing-n:127.0.0.1:50535_) [n:127.0.0.1:50535_ c:collection1   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0
>    [junit4]   2> 2808997 INFO  (coreLoadExecutor-5959-thread-1-processing-n:127.0.0.1:50535_) [n:127.0.0.1:50535_ c:collection1   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
>    [junit4]   2> 2809010 INFO  (coreLoadExecutor-5959-thread-1-processing-n:127.0.0.1:50535_) [n:127.0.0.1:50535_ c:collection1   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
>    [junit4]   2> 2809039 INFO  (coreLoadExecutor-5959-thread-1-processing-n:127.0.0.1:50535_) [n:127.0.0.1:50535_ c:collection1   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
>    [junit4]   2> 2809040 INFO  (coreLoadExecutor-5959-thread-1-processing-n:127.0.0.1:50535_) [n:127.0.0.1:50535_ c:collection1   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
>    [junit4]   2> 2809046 INFO  (coreLoadExecutor-5959-thread-1-processing-n:127.0.0.1:50535_) [n:127.0.0.1:50535_ c:collection1   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
>    [junit4]   2> 2809381 INFO  (coreLoadExecutor-5959-thread-1-processing-n:127.0.0.1:50535_) [n:127.0.0.1:50535_ c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
>    [junit4]   2> 2809388 INFO  (coreLoadExecutor-5959-thread-1-processing-n:127.0.0.1:50535_) [n:127.0.0.1:50535_ c:collection1   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
>    [junit4]   2> 2809390 INFO  (coreLoadExecutor-5959-thread-1-processing-n:127.0.0.1:50535_) [n:127.0.0.1:50535_ c:collection1   x:collection1] o.a.s.s.IndexSchema unique key field: id
>    [junit4]   2> 2809400 INFO  (coreLoadExecutor-5959-thread-
>
> [...truncated too long message...]
>
> ill nodes recoverying - waited for 330 seconds
>    [junit4]    >        at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:172)
>    [junit4]    >        at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:133)
>    [junit4]    >        at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:128)
>    [junit4]    >        at org.apache.solr.cloud.AbstractFullDistribZkTestBase.waitForRecoveriesToFinish(AbstractFullDistribZkTestBase.java:830)
>    [junit4]    >        at org.apache.solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR(LeaderInitiatedRecoveryOnShardRestartTest.java:81)
>    [junit4]    >        at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:963)
>    [junit4]    >        at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:938)
>    [junit4]    >        at java.lang.Thread.run(Thread.java:745)Throwable #2: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=43491, name=coreZkRegister-5997-thread-1, state=RUNNABLE, group=TGRP-LeaderInitiatedRecoveryOnShardRestartTest]
>    [junit4]    > Caused by: java.lang.AssertionError
>    [junit4]    >        at __randomizedtesting.SeedInfo.seed([7F78F76DDF75FAD1]:0)
>    [junit4]    >        at org.apache.solr.cloud.ZkController.updateLeaderInitiatedRecoveryState(ZkController.java:2133)
>    [junit4]    >        at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:434)
>    [junit4]    >        at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)
>    [junit4]    >        at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)
>    [junit4]    >        at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:346)
>    [junit4]    >        at org.apache.solr.cloud.ZkController.joinElection(ZkController.java:1113)
>    [junit4]    >        at org.apache.solr.cloud.ZkController.register(ZkController.java:926)
>    [junit4]    >        at org.apache.solr.cloud.ZkController.register(ZkController.java:881)
>    [junit4]    >        at org.apache.solr.core.ZkContainer$2.run(ZkContainer.java:183)
>    [junit4]    >        at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)
>    [junit4]    >        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>    [junit4]    >        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>    [junit4]    >        at java.lang.Thread.run(Thread.java:745)
>    [junit4]   2> 3169248 INFO  (SUITE-LeaderInitiatedRecoveryOnShardRestartTest-seed#[7F78F76DDF75FAD1]-worker) [n:127.0.0.1:33453_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.SolrTestCaseJ4 ###deleteCore
>    [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001
>    [junit4]   2> Oct 22, 2015 2:22:32 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
>    [junit4]   2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
>    [junit4]   2> NOTE: test params are: codec=Asserting(Lucene53), sim=RandomSimilarityProvider(queryNorm=true,coord=yes): {}, locale=be_BY, timezone=Pacific/Port_Moresby
>    [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_72 (64-bit)/cpus=4,threads=1,free=204385304,total=520093696
>    [junit4]   2> NOTE: All tests run in this JVM: [TestExceedMaxTermLength, BinaryUpdateRequestHandlerTest, TestAuthorizationFramework, TestNamedUpdateProcessors, CoreAdminCreateDiscoverTest, TestCloudManagedSchema, TestRemoveLastDynamicCopyField, TestRawResponseWriter, TestCloudSchemaless, ZkStateReaderTest, TestZkChroot, TestSolrQueryParser, TestReload, TestReversedWildcardFilterFactory, HttpPartitionTest, TestSchemaManager, TestPostingsSolrHighlighter, HdfsLockFactoryTest, QueryParsingTest, TestIndexingPerformance, TestCloudManagedSchemaConcurrent, RAMDirectoryFactoryTest, CloudExitableDirectoryReaderTest, CoreAdminHandlerTest, VMParamsZkACLAndCredentialsProvidersTest, SampleTest, AutoCommitTest, TestSolrDeletionPolicy1, StatsComponentTest, SolrCoreCheckLockOnStartupTest, AsyncMigrateRouteKeyTest, TestSimpleQParserPlugin, SimpleCollectionCreateDeleteTest, SuggesterWFSTTest, VersionInfoTest, ChangedSchemaMergeTest, TestPKIAuthenticationPlugin, TestIndexSearcher, DistributedQueryComponentCustomSortTest, TestMacros, PrimitiveFieldTypeTest, DeleteShardTest, PluginInfoTest, HdfsRecoveryZkTest, SSLMigrationTest, DistributedFacetPivotSmallAdvancedTest, RequestHandlersTest, BadComponentTest, TestSearcherReuse, DocValuesMultiTest, ConnectionManagerTest, FullSolrCloudDistribCmdsTest, AnalyticsMergeStrategyTest, DeleteLastCustomShardedReplicaTest, TestConfigSets, TestCustomSort, TestMaxScoreQueryParser, CollectionReloadTest, TestInitParams, ExitableDirectoryReaderTest, DistribCursorPagingTest, DocValuesMissingTest, DebugComponentTest, TestSystemIdResolver, AnalysisErrorHandlingTest, TestSortByMinMaxFunction, DirectUpdateHandlerOptimizeTest, TestFieldTypeCollectionResource, SearchHandlerTest, TestElisionMultitermQuery, TestManagedStopFilterFactory, TestJmxIntegration, RulesTest, HighlighterMaxOffsetTest, TestCollationField, LukeRequestHandlerTest, TestDocBasedVersionConstraints, TestDefaultSearchFieldResource, FieldAnalysisRequestHandlerTest, FieldMutatingUpdateProcessorTest, TestPivotHelperCode, DocExpirationUpdateProcessorFactoryTest, StressHdfsTest, ScriptEngineTest, ChaosMonkeySafeLeaderTest, BasicDistributedZk2Test, UnloadDistributedZkTest, SyncSliceTest, LeaderElectionIntegrationTest, BasicZkTest, RecoveryZkTest, TestRandomFaceting, ShardRoutingCustomTest, TestDistributedGrouping, TestFaceting, TestRecovery, TestHashPartitioner, TermVectorComponentDistributedTest, TestStressReorder, TestJoin, TestStressVersions, HardAutoCommitTest, DistributedTermsComponentTest, TestRangeQuery, TestCoreContainer, SimpleFacetsTest, TestSolr4Spatial, TestGroupingSearch, PeerSyncTest, BadIndexSchemaTest, ConvertedLegacyTest, TestSort, BasicFunctionalityTest, DirectUpdateHandlerTest, TestBadConfig, SoftAutoCommitTest, ShowFileRequestHandlerTest, CurrencyFieldOpenExchangeTest, CurrencyFieldXmlFileTest, SolrIndexSplitterTest, TestCoreDiscovery, AnalysisAfterCoreReloadTest, SignatureUpdateProcessorFactoryTest, TestExtendedDismaxParser, SuggesterTSTTest, SuggesterTest, TestStressLucene, TestTrie, PolyFieldTest, NoCacheHeaderTest, WordBreakSolrSpellCheckerTest, SchemaVersionSpecificBehaviorTest, TestPseudoReturnFields, TestUpdate, TestWordDelimiterFilterFactory, QueryEqualityTest, DefaultValueUpdateProcessorTest, StatelessScriptUpdateProcessorFactoryTest, SortByFunctionTest, XsltUpdateRequestHandlerTest, IndexBasedSpellCheckerTest, DisMaxRequestHandlerTest, TestWriterPerf, DirectSolrSpellCheckerTest, DocumentAnalysisRequestHandlerTest, TestQueryTypes, TestOmitPositions, FileBasedSpellCheckerTest, TermsComponentTest, XmlUpdateRequestHandlerTest, DocumentBuilderTest, TestValueSourceCache, TermVectorComponentTest, RequiredFieldsTest, TestArbitraryIndexDir, FastVectorHighlighterTest, IndexSchemaRuntimeFieldTest, LoggingHandlerTest, RegexBoostProcessorTest, SolrPluginUtilsTest, ReturnFieldsTest, JSONWriterTest, TestCSVResponseWriter, MBeansHandlerTest, TestAnalyzedSuggestions, UniqFieldsUpdateProcessorFactoryTest, TestPHPSerializedResponseWriter, HighlighterConfigTest, SOLR749Test, UpdateParamsTest, TestSolrIndexConfig, SolrIndexConfigTest, TestStressRecovery, MultiTermTest, TestBinaryField, TestSearchPerf, NumericFieldsTest, OutputWriterTest, TestFuzzyAnalyzedSuggestions, TestCharFilters, TestSweetSpotSimilarityFactory, TestDFRSimilarityFactory, TestLMDirichletSimilarityFactory, TestFastWriter, TestFastOutputStream, URLClassifyProcessorTest, DateMathParserTest, PrimUtilsTest, SpellingQueryConverterTest, ClusterStateTest, TestLRUCache, TestDocumentBuilder, SystemInfoHandlerTest, FileUtilsTest, DistributedMLTComponentTest, TestCrossCoreJoin, TestCursorMarkWithoutUniqueKey, TestSimpleTrackingShardHandler, TestTolerantSearch, TestEmbeddedSolrServerConstructors, ActionThrottleTest, AsyncCallRequestStatusResponseTest, CleanupOldIndexTest, CollectionStateFormat2Test, DeleteInactiveReplicaTest, DistribDocExpirationUpdateProcessorTest, LeaderInitiatedRecoveryOnShardRestartTest]
>    [junit4] Completed [439/542] on J2 in 364.94s, 1 test, 1 error <<< FAILURES!
>
> [...truncated 365 lines...]
> BUILD FAILED
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 542 suites (7 ignored), 2143 tests, 1 error, 31 ignored (18 assumptions) [seed: 7F78F76DDF75FAD1]
>
> Total time: 198 minutes 25 seconds
> Build step 'Invoke Ant' marked build as failure
> Archiving artifacts
> Recording test results
> Email was triggered for: Failure - Any
> Sending email for trigger: Failure - Any
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org



-- 
Regards,
Shalin Shekhar Mangar.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org

[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 993 - Still Failing

[Apache Jenkins Server <je...@builds.apache.org>]

Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/993/

1 tests failed.
FAILED:  org.apache.solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR

Error Message:
Captured an uncaught exception in thread: Thread[id=43491, name=coreZkRegister-5997-thread-1, state=RUNNABLE, group=TGRP-LeaderInitiatedRecoveryOnShardRestartTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=43491, name=coreZkRegister-5997-thread-1, state=RUNNABLE, group=TGRP-LeaderInitiatedRecoveryOnShardRestartTest]
Caused by: java.lang.AssertionError
	at __randomizedtesting.SeedInfo.seed([7F78F76DDF75FAD1]:0)
	at org.apache.solr.cloud.ZkController.updateLeaderInitiatedRecoveryState(ZkController.java:2133)
	at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:434)
	at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)
	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)
	at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:346)
	at org.apache.solr.cloud.ZkController.joinElection(ZkController.java:1113)
	at org.apache.solr.cloud.ZkController.register(ZkController.java:926)
	at org.apache.solr.cloud.ZkController.register(ZkController.java:881)
	at org.apache.solr.core.ZkContainer$2.run(ZkContainer.java:183)
	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 11227 lines...]
   [junit4] Suite: org.apache.solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/init-core-data-001
   [junit4]   2> 2804605 INFO  (SUITE-LeaderInitiatedRecoveryOnShardRestartTest-seed#[7F78F76DDF75FAD1]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   2> 2804608 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 2804609 INFO  (Thread-33860) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 2804609 INFO  (Thread-33860) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 2804709 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.ZkTestServer start zk server on port:52462
   [junit4]   2> 2804709 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2804709 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2804712 INFO  (zkCallback-2280-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7a70d2b6 name:ZooKeeperConnection Watcher:127.0.0.1:52462 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2804712 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2804712 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2804712 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 2804715 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2804715 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2804716 INFO  (zkCallback-2281-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5128e657 name:ZooKeeperConnection Watcher:127.0.0.1:52462/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2804716 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2804716 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2804716 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 2804718 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 2804719 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 2804720 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 2804722 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 2804722 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 2804724 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 2804724 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 2804726 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 2804726 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 2804728 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 2804728 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 2804729 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 2804730 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 2804731 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 2804732 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 2804733 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 2804733 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 2804738 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 2804738 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 2804739 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 2804740 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 2804741 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 2804741 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 2804743 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 2804743 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 2805008 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/control-001/cores/collection1
   [junit4]   2> 2805010 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 2805012 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@665b505c{/,null,AVAILABLE}
   [junit4]   2> 2805013 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.e.j.s.ServerConnector Started ServerConnector@68a3874{HTTP/1.1}{127.0.0.1:56746}
   [junit4]   2> 2805013 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.e.j.s.Server Started @2807895ms
   [junit4]   2> 2805013 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostPort=56746, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/tempDir-001/control/data, hostContext=/, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/control-001/cores}
   [junit4]   2> 2805013 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@1fe91485
   [junit4]   2> 2805013 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/control-001/'
   [junit4]   2> 2805013 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 2805013 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 2805027 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2805028 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2805029 INFO  (zkCallback-2282-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@39baa32 name:ZooKeeperConnection Watcher:127.0.0.1:52462/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2805029 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2805029 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2805031 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 2805031 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/control-001/solr.xml
   [junit4]   2> 2805043 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/control-001/cores
   [junit4]   2> 2805043 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.CoreContainer New CoreContainer 1984045032
   [junit4]   2> 2805043 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/control-001/]
   [junit4]   2> 2805044 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/control-001/lib
   [junit4]   2> 2805044 WARN  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/control-001/lib).
   [junit4]   2> 2805051 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 2805052 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 2805052 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 2805052 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 2805052 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:52462/solr
   [junit4]   2> 2805052 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 2805052 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2805054 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2805054 INFO  (zkCallback-2284-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2e9d74e9 name:ZooKeeperConnection Watcher:127.0.0.1:52462 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2805054 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2805055 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2805056 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2805057 INFO  (zkCallback-2285-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@51ed1947 name:ZooKeeperConnection Watcher:127.0.0.1:52462/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2805058 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2805058 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 2805061 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 2805063 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 2805065 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 2805066 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 2805068 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 2805069 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 2805070 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 2805071 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 2805072 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:56746_
   [junit4]   2> 2805073 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:56746_
   [junit4]   2> 2805074 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 2805075 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 2805076 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 2805077 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94731933327425540-127.0.0.1:56746_-n_0000000000
   [junit4]   2> 2805077 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:56746_
   [junit4]   2> 2805077 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 2805078 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.c.Overseer Overseer (id=94731933327425540-127.0.0.1:56746_-n_0000000000) starting
   [junit4]   2> 2805080 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 2805085 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 2805085 INFO  (OverseerCollectionConfigSetProcessor-94731933327425540-127.0.0.1:56746_-n_0000000000) [n:127.0.0.1:56746_    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 2805085 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 2805085 INFO  (OverseerStateUpdate-94731933327425540-127.0.0.1:56746_-n_0000000000) [n:127.0.0.1:56746_    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 2805108 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 2805109 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 2805110 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/control-001/cores
   [junit4]   2> 2805110 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.c.SolrCore Created CoreDescriptor: {loadOnStartup=true, configSetProperties=configsetprops.json, coreNodeName=, dataDir=data/, name=collection1, config=solrconfig.xml, collection=control_collection, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/control-001/cores/collection1/, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/control-001/cores/collection1, shard=, transient=false, schema=schema.xml}
   [junit4]   2> 2805110 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/control-001/cores/collection1/
   [junit4]   2> 2805111 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 2805111 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 2805111 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:56746_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 2805112 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 2805112 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 2805113 INFO  (OverseerStateUpdate-94731933327425540-127.0.0.1:56746_-n_0000000000) [n:127.0.0.1:56746_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:56746",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "node_name":"127.0.0.1:56746_"} current state version: 0
   [junit4]   2> 2805113 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 2805114 INFO  (OverseerStateUpdate-94731933327425540-127.0.0.1:56746_-n_0000000000) [n:127.0.0.1:56746_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:56746",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "node_name":"127.0.0.1:56746_"}
   [junit4]   2> 2805114 INFO  (OverseerStateUpdate-94731933327425540-127.0.0.1:56746_-n_0000000000) [n:127.0.0.1:56746_    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 2805114 INFO  (OverseerStateUpdate-94731933327425540-127.0.0.1:56746_-n_0000000000) [n:127.0.0.1:56746_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 2805116 INFO  (zkCallback-2285-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2806114 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 2806114 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 2806115 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 2806115 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 2806115 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 2806116 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/control-001/cores/collection1/'
   [junit4]   2> 2806116 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 2806116 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 2806129 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 2806133 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 2806145 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 2806170 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 2806171 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 2806177 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 2806414 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 2806439 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 2806440 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 2806448 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 2806451 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 2806454 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 2806454 WARN  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 2806454 WARN  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 2806454 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 2806455 WARN  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 2806455 WARN  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 2806456 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 2806456 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 2806456 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 2806456 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/control-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 2806456 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@379f19b4
   [junit4]   2> 2806457 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/control-001/cores/collection1/data
   [junit4]   2> 2806457 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/control-001/cores/collection1/data/index/
   [junit4]   2> 2806457 WARN  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 2806457 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/control-001/cores/collection1/data/index
   [junit4]   2> 2806458 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@7cd8dcec
   [junit4]   2> 2806458 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@19049fd lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@27e981d8),segFN=segments_1,generation=1}
   [junit4]   2> 2806458 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 2806458 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/control-001/cores/collection1/data/
   [junit4]   2> 2806459 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 2806462 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 2806462 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 2806462 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 2806462 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 2806462 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 2806462 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 2806462 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 2806462 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 2806463 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 2806465 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2806466 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2806468 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2806469 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2806472 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/logging,/update/json,/admin/segments,/admin/mbeans,/admin/ping,/update/csv,/admin/plugins,/update,/admin/file,/replication,standard,/update/json/docs,/admin/threads,/admin/properties,/admin/luke,/admin/system,/get,/schema,/config
   [junit4]   2> 2806473 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 2806475 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 2806475 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 2806475 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 2806476 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 2806476 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=12, maxMergeAtOnceExplicit=36, maxMergedSegmentMB=82.4736328125, floorSegmentMB=1.5283203125, forceMergeDeletesPctAllowed=4.112648727330555, segmentsPerTier=14.0, maxCFSSegmentSizeMB=1.6865234375, noCFSRatio=0.0
   [junit4]   2> 2806476 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@19049fd lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@27e981d8),segFN=segments_1,generation=1}
   [junit4]   2> 2806476 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 2806476 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@772028e6[collection1] main
   [junit4]   2> 2806477 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 2806478 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 2806478 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 2806478 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 2806478 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 2806478 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 2806479 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 2806479 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 2806479 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 2806479 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 2806479 INFO  (searcherExecutor-5949-thread-1-processing-r:core_node1 s:shard1 x:collection1 c:control_collection n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@772028e6[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2806479 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 2806479 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 2806479 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 2806480 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1515710935199645696
   [junit4]   2> 2806482 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 2.0ms to seed version buckets with highest version 1515710935199645696
   [junit4]   2> 2806482 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 2806482 INFO  (coreLoadExecutor-5948-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 2806482 INFO  (coreZkRegister-5942-thread-1-processing-r:core_node1 s:shard1 x:collection1 c:control_collection n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:56746 collection:control_collection shard:shard1
   [junit4]   2> 2806483 INFO  (coreZkRegister-5942-thread-1-processing-r:core_node1 s:shard1 x:collection1 c:control_collection n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 2806486 INFO  (coreZkRegister-5942-thread-1-processing-r:core_node1 s:shard1 x:collection1 c:control_collection n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94731933327425540-core_node1-n_0000000000
   [junit4]   2> 2806487 INFO  (coreZkRegister-5942-thread-1-processing-r:core_node1 s:shard1 x:collection1 c:control_collection n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 2806488 INFO  (coreZkRegister-5942-thread-1-processing-r:core_node1 s:shard1 x:collection1 c:control_collection n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 2806488 INFO  (coreZkRegister-5942-thread-1-processing-r:core_node1 s:shard1 x:collection1 c:control_collection n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 2806488 INFO  (coreZkRegister-5942-thread-1-processing-r:core_node1 s:shard1 x:collection1 c:control_collection n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:56746/collection1/
   [junit4]   2> 2806488 INFO  (coreZkRegister-5942-thread-1-processing-r:core_node1 s:shard1 x:collection1 c:control_collection n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 2806488 INFO  (OverseerStateUpdate-94731933327425540-127.0.0.1:56746_-n_0000000000) [n:127.0.0.1:56746_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 2806488 INFO  (coreZkRegister-5942-thread-1-processing-r:core_node1 s:shard1 x:collection1 c:control_collection n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:56746/collection1/ has no replicas
   [junit4]   2> 2806488 INFO  (coreZkRegister-5942-thread-1-processing-r:core_node1 s:shard1 x:collection1 c:control_collection n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:56746/collection1/ shard1
   [junit4]   2> 2806489 INFO  (coreZkRegister-5942-thread-1-processing-r:core_node1 s:shard1 x:collection1 c:control_collection n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 2806491 INFO  (coreZkRegister-5942-thread-1-processing-r:core_node1 s:shard1 x:collection1 c:control_collection n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 2806493 INFO  (OverseerStateUpdate-94731933327425540-127.0.0.1:56746_-n_0000000000) [n:127.0.0.1:56746_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:56746",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 1
   [junit4]   2> 2806595 INFO  (zkCallback-2285-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2806613 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2806614 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2806615 INFO  (zkCallback-2287-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@794415fc name:ZooKeeperConnection Watcher:127.0.0.1:52462/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2806615 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2806615 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2806615 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 2806617 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 2806617 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with stateFormat=2
   [junit4]   2> 2806617 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2806618 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2806619 INFO  (zkCallback-2288-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@77c64dbb name:ZooKeeperConnection Watcher:127.0.0.1:52462/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2806619 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2806619 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2806622 INFO  (OverseerStateUpdate-94731933327425540-127.0.0.1:56746_-n_0000000000) [n:127.0.0.1:56746_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"create",
   [junit4]   2>   "name":"collection1",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "stateFormat":"2"} current state version: 2
   [junit4]   2> 2806622 INFO  (OverseerStateUpdate-94731933327425540-127.0.0.1:56746_-n_0000000000) [n:127.0.0.1:56746_    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 2806644 INFO  (coreZkRegister-5942-thread-1-processing-r:core_node1 s:shard1 x:collection1 c:control_collection n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:56746/collection1/ and leader is http://127.0.0.1:56746/collection1/
   [junit4]   2> 2806644 INFO  (coreZkRegister-5942-thread-1-processing-r:core_node1 s:shard1 x:collection1 c:control_collection n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:56746
   [junit4]   2> 2806644 INFO  (coreZkRegister-5942-thread-1-processing-r:core_node1 s:shard1 x:collection1 c:control_collection n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 2806644 INFO  (coreZkRegister-5942-thread-1-processing-r:core_node1 s:shard1 x:collection1 c:control_collection n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 2806644 INFO  (coreZkRegister-5942-thread-1-processing-r:core_node1 s:shard1 x:collection1 c:control_collection n:127.0.0.1:56746_) [n:127.0.0.1:56746_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 2806646 INFO  (OverseerStateUpdate-94731933327425540-127.0.0.1:56746_-n_0000000000) [n:127.0.0.1:56746_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:56746",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "node_name":"127.0.0.1:56746_"} current state version: 2
   [junit4]   2> 2806646 INFO  (OverseerStateUpdate-94731933327425540-127.0.0.1:56746_-n_0000000000) [n:127.0.0.1:56746_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:56746",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "node_name":"127.0.0.1:56746_"}
   [junit4]   2> 2806647 INFO  (OverseerStateUpdate-94731933327425540-127.0.0.1:56746_-n_0000000000) [n:127.0.0.1:56746_    ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/collection1/state.json
   [junit4]   2> 2806750 INFO  (zkCallback-2287-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2806750 INFO  (zkCallback-2285-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2806891 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/shard-1-001/cores/collection1
   [junit4]   2> 2806891 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/shard-1-001
   [junit4]   2> 2806892 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 2806894 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@719f3138{/,null,AVAILABLE}
   [junit4]   2> 2806894 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.e.j.s.ServerConnector Started ServerConnector@3bc61657{HTTP/1.1}{127.0.0.1:50535}
   [junit4]   2> 2806894 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.e.j.s.Server Started @2809776ms
   [junit4]   2> 2806894 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/shard-1-001/cores, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/tempDir-001/jetty1, hostContext=/, solrconfig=solrconfig.xml, hostPort=50535}
   [junit4]   2> 2806895 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@1fe91485
   [junit4]   2> 2806895 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/shard-1-001/'
   [junit4]   2> 2806895 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 2806895 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 2806909 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2806910 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2806911 INFO  (zkCallback-2289-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@306792d6 name:ZooKeeperConnection Watcher:127.0.0.1:52462/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2806911 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2806911 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2806913 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 2806913 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/shard-1-001/solr.xml
   [junit4]   2> 2806926 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/shard-1-001/cores
   [junit4]   2> 2806926 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.CoreContainer New CoreContainer 1559122149
   [junit4]   2> 2806926 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/shard-1-001/]
   [junit4]   2> 2806926 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/shard-1-001/lib
   [junit4]   2> 2806926 WARN  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/shard-1-001/lib).
   [junit4]   2> 2806934 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 2806935 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 2806935 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 2806935 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 2806935 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:52462/solr
   [junit4]   2> 2806935 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 2806936 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2806936 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2806937 INFO  (zkCallback-2291-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@4c57b1e6 name:ZooKeeperConnection Watcher:127.0.0.1:52462 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2806937 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2806938 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2806940 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:50535_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2806941 INFO  (zkCallback-2292-thread-1-processing-n:127.0.0.1:50535_) [n:127.0.0.1:50535_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@124679be name:ZooKeeperConnection Watcher:127.0.0.1:52462/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2806941 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:50535_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2806945 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:50535_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 2807951 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:50535_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50535_
   [junit4]   2> 2807951 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:50535_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:50535_
   [junit4]   2> 2807952 INFO  (zkCallback-2287-thread-1) [    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2807952 INFO  (zkCallback-2285-thread-1-processing-n:127.0.0.1:56746_) [n:127.0.0.1:56746_    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2807952 INFO  (zkCallback-2292-thread-1-processing-n:127.0.0.1:50535_) [n:127.0.0.1:50535_    ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2807954 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:50535_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 2807955 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:50535_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94731933327425545-127.0.0.1:50535_-n_0000000001
   [junit4]   2> 2807955 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:50535_    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/94731933327425540-127.0.0.1:56746_-n_0000000000 to know if I could be the leader
   [junit4]   2> 2807971 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:50535_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 2807971 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:50535_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 2807972 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:50535_    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/shard-1-001/cores
   [junit4]   2> 2807972 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:50535_    ] o.a.s.c.SolrCore Created CoreDescriptor: {collection=collection1, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/shard-1-001/cores/collection1/, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/shard-1-001/cores/collection1, name=collection1, configSetProperties=configsetprops.json, shard=, coreNodeName=, loadOnStartup=true, dataDir=data/, schema=schema.xml, transient=false, config=solrconfig.xml}
   [junit4]   2> 2807973 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:50535_    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/shard-1-001/cores/collection1/
   [junit4]   2> 2807973 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:50535_    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 2807974 INFO  (coreLoadExecutor-5959-thread-1-processing-n:127.0.0.1:50535_) [n:127.0.0.1:50535_ c:collection1   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 2807974 INFO  (coreLoadExecutor-5959-thread-1-processing-n:127.0.0.1:50535_) [n:127.0.0.1:50535_ c:collection1   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 2807974 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:50535_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2
   [junit4]   2> 2807974 INFO  (TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[7F78F76DDF75FAD1]) [n:127.0.0.1:50535_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 2807975 INFO  (coreLoadExecutor-5959-thread-1-processing-n:127.0.0.1:50535_) [n:127.0.0.1:50535_ c:collection1   x:collection1] o.a.s.c.ZkController Registering watch for collection collection1
   [junit4]   2> 2807975 INFO  (coreLoadExecutor-5959-thread-1-processing-n:127.0.0.1:50535_) [n:127.0.0.1:50535_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader addZkWatch collection1
   [junit4]   2> 2807976 INFO  (OverseerStateUpdate-94731933327425540-127.0.0.1:56746_-n_0000000000) [n:127.0.0.1:56746_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:50535",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "node_name":"127.0.0.1:50535_"} current state version: 3
   [junit4]   2> 2807976 INFO  (OverseerStateUpdate-94731933327425540-127.0.0.1:56746_-n_0000000000) [n:127.0.0.1:56746_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:50535",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "node_name":"127.0.0.1:50535_"}
   [junit4]   2> 2807976 INFO  (OverseerStateUpdate-94731933327425540-127.0.0.1:56746_-n_0000000000) [n:127.0.0.1:56746_    ] o.a.s.c.o.ReplicaMutator Collection already exists with numShards=1
   [junit4]   2> 2807976 INFO  (OverseerStateUpdate-94731933327425540-127.0.0.1:56746_-n_0000000000) [n:127.0.0.1:56746_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 2807976 INFO  (coreLoadExecutor-5959-thread-1-processing-n:127.0.0.1:50535_) [n:127.0.0.1:50535_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Add data for collection1 ver 0 
   [junit4]   2> 2807977 INFO  (coreLoadExecutor-5959-thread-1-processing-n:127.0.0.1:50535_) [n:127.0.0.1:50535_ c:collection1   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 2808078 INFO  (OverseerStateUpdate-94731933327425540-127.0.0.1:56746_-n_0000000000) [n:127.0.0.1:56746_    ] o.a.s.c.o.ZkStateWriter going to update_collection /collections/collection1/state.json version: 0
   [junit4]   2> 2808079 INFO  (zkCallback-2292-thread-1-processing-n:127.0.0.1:50535_) [n:127.0.0.1:50535_    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json for collection collection1 has occurred - updating... (live nodes size: 2)
   [junit4]   2> 2808079 INFO  (zkCallback-2292-thread-1-processing-n:127.0.0.1:50535_) [n:127.0.0.1:50535_    ] o.a.s.c.c.ZkStateReader Updating data for collection1 from 0 to 1 
   [junit4]   2> 2808977 INFO  (coreLoadExecutor-5959-thread-1-processing-n:127.0.0.1:50535_) [n:127.0.0.1:50535_ c:collection1   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 2808977 INFO  (coreLoadExecutor-5959-thread-1-processing-n:127.0.0.1:50535_) [n:127.0.0.1:50535_ c:collection1   x:collection1] o.a.s.c.ZkController Check for collection zkNode:collection1
   [junit4]   2> 2808978 INFO  (coreLoadExecutor-5959-thread-1-processing-n:127.0.0.1:50535_) [n:127.0.0.1:50535_ c:collection1   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 2808978 INFO  (coreLoadExecutor-5959-thread-1-processing-n:127.0.0.1:50535_) [n:127.0.0.1:50535_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/collection1
   [junit4]   2> 2808978 INFO  (coreLoadExecutor-5959-thread-1-processing-n:127.0.0.1:50535_) [n:127.0.0.1:50535_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 2808978 INFO  (coreLoadExecutor-5959-thread-1-processing-n:127.0.0.1:50535_) [n:127.0.0.1:50535_ c:collection1   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001/shard-1-001/cores/collection1/'
   [junit4]   2> 2808979 INFO  (coreLoadExecutor-5959-thread-1-processing-n:127.0.0.1:50535_) [n:127.0.0.1:50535_ c:collection1   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 2808979 INFO  (coreLoadExecutor-5959-thread-1-processing-n:127.0.0.1:50535_) [n:127.0.0.1:50535_ c:collection1   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 2808992 INFO  (coreLoadExecutor-5959-thread-1-processing-n:127.0.0.1:50535_) [n:127.0.0.1:50535_ c:collection1   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 2808997 INFO  (coreLoadExecutor-5959-thread-1-processing-n:127.0.0.1:50535_) [n:127.0.0.1:50535_ c:collection1   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 2809010 INFO  (coreLoadExecutor-5959-thread-1-processing-n:127.0.0.1:50535_) [n:127.0.0.1:50535_ c:collection1   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
   [junit4]   2> 2809039 INFO  (coreLoadExecutor-5959-thread-1-processing-n:127.0.0.1:50535_) [n:127.0.0.1:50535_ c:collection1   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 2809040 INFO  (coreLoadExecutor-5959-thread-1-processing-n:127.0.0.1:50535_) [n:127.0.0.1:50535_ c:collection1   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 2809046 INFO  (coreLoadExecutor-5959-thread-1-processing-n:127.0.0.1:50535_) [n:127.0.0.1:50535_ c:collection1   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 2809381 INFO  (coreLoadExecutor-5959-thread-1-processing-n:127.0.0.1:50535_) [n:127.0.0.1:50535_ c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 2809388 INFO  (coreLoadExecutor-5959-thread-1-processing-n:127.0.0.1:50535_) [n:127.0.0.1:50535_ c:collection1   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 2809390 INFO  (coreLoadExecutor-5959-thread-1-processing-n:127.0.0.1:50535_) [n:127.0.0.1:50535_ c:collection1   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 2809400 INFO  (coreLoadExecutor-5959-thread-

[...truncated too long message...]

ill nodes recoverying - waited for 330 seconds
   [junit4]    > 	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:172)
   [junit4]    > 	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:133)
   [junit4]    > 	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:128)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.waitForRecoveriesToFinish(AbstractFullDistribZkTestBase.java:830)
   [junit4]    > 	at org.apache.solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR(LeaderInitiatedRecoveryOnShardRestartTest.java:81)
   [junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:963)
   [junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:938)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)Throwable #2: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=43491, name=coreZkRegister-5997-thread-1, state=RUNNABLE, group=TGRP-LeaderInitiatedRecoveryOnShardRestartTest]
   [junit4]    > Caused by: java.lang.AssertionError
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([7F78F76DDF75FAD1]:0)
   [junit4]    > 	at org.apache.solr.cloud.ZkController.updateLeaderInitiatedRecoveryState(ZkController.java:2133)
   [junit4]    > 	at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:434)
   [junit4]    > 	at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)
   [junit4]    > 	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)
   [junit4]    > 	at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:346)
   [junit4]    > 	at org.apache.solr.cloud.ZkController.joinElection(ZkController.java:1113)
   [junit4]    > 	at org.apache.solr.cloud.ZkController.register(ZkController.java:926)
   [junit4]    > 	at org.apache.solr.cloud.ZkController.register(ZkController.java:881)
   [junit4]    > 	at org.apache.solr.core.ZkContainer$2.run(ZkContainer.java:183)
   [junit4]    > 	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)
   [junit4]    > 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   [junit4]    > 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 3169248 INFO  (SUITE-LeaderInitiatedRecoveryOnShardRestartTest-seed#[7F78F76DDF75FAD1]-worker) [n:127.0.0.1:33453_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_7F78F76DDF75FAD1-001
   [junit4]   2> Oct 22, 2015 2:22:32 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene53), sim=RandomSimilarityProvider(queryNorm=true,coord=yes): {}, locale=be_BY, timezone=Pacific/Port_Moresby
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_72 (64-bit)/cpus=4,threads=1,free=204385304,total=520093696
   [junit4]   2> NOTE: All tests run in this JVM: [TestExceedMaxTermLength, BinaryUpdateRequestHandlerTest, TestAuthorizationFramework, TestNamedUpdateProcessors, CoreAdminCreateDiscoverTest, TestCloudManagedSchema, TestRemoveLastDynamicCopyField, TestRawResponseWriter, TestCloudSchemaless, ZkStateReaderTest, TestZkChroot, TestSolrQueryParser, TestReload, TestReversedWildcardFilterFactory, HttpPartitionTest, TestSchemaManager, TestPostingsSolrHighlighter, HdfsLockFactoryTest, QueryParsingTest, TestIndexingPerformance, TestCloudManagedSchemaConcurrent, RAMDirectoryFactoryTest, CloudExitableDirectoryReaderTest, CoreAdminHandlerTest, VMParamsZkACLAndCredentialsProvidersTest, SampleTest, AutoCommitTest, TestSolrDeletionPolicy1, StatsComponentTest, SolrCoreCheckLockOnStartupTest, AsyncMigrateRouteKeyTest, TestSimpleQParserPlugin, SimpleCollectionCreateDeleteTest, SuggesterWFSTTest, VersionInfoTest, ChangedSchemaMergeTest, TestPKIAuthenticationPlugin, TestIndexSearcher, DistributedQueryComponentCustomSortTest, TestMacros, PrimitiveFieldTypeTest, DeleteShardTest, PluginInfoTest, HdfsRecoveryZkTest, SSLMigrationTest, DistributedFacetPivotSmallAdvancedTest, RequestHandlersTest, BadComponentTest, TestSearcherReuse, DocValuesMultiTest, ConnectionManagerTest, FullSolrCloudDistribCmdsTest, AnalyticsMergeStrategyTest, DeleteLastCustomShardedReplicaTest, TestConfigSets, TestCustomSort, TestMaxScoreQueryParser, CollectionReloadTest, TestInitParams, ExitableDirectoryReaderTest, DistribCursorPagingTest, DocValuesMissingTest, DebugComponentTest, TestSystemIdResolver, AnalysisErrorHandlingTest, TestSortByMinMaxFunction, DirectUpdateHandlerOptimizeTest, TestFieldTypeCollectionResource, SearchHandlerTest, TestElisionMultitermQuery, TestManagedStopFilterFactory, TestJmxIntegration, RulesTest, HighlighterMaxOffsetTest, TestCollationField, LukeRequestHandlerTest, TestDocBasedVersionConstraints, TestDefaultSearchFieldResource, FieldAnalysisRequestHandlerTest, FieldMutatingUpdateProcessorTest, TestPivotHelperCode, DocExpirationUpdateProcessorFactoryTest, StressHdfsTest, ScriptEngineTest, ChaosMonkeySafeLeaderTest, BasicDistributedZk2Test, UnloadDistributedZkTest, SyncSliceTest, LeaderElectionIntegrationTest, BasicZkTest, RecoveryZkTest, TestRandomFaceting, ShardRoutingCustomTest, TestDistributedGrouping, TestFaceting, TestRecovery, TestHashPartitioner, TermVectorComponentDistributedTest, TestStressReorder, TestJoin, TestStressVersions, HardAutoCommitTest, DistributedTermsComponentTest, TestRangeQuery, TestCoreContainer, SimpleFacetsTest, TestSolr4Spatial, TestGroupingSearch, PeerSyncTest, BadIndexSchemaTest, ConvertedLegacyTest, TestSort, BasicFunctionalityTest, DirectUpdateHandlerTest, TestBadConfig, SoftAutoCommitTest, ShowFileRequestHandlerTest, CurrencyFieldOpenExchangeTest, CurrencyFieldXmlFileTest, SolrIndexSplitterTest, TestCoreDiscovery, AnalysisAfterCoreReloadTest, SignatureUpdateProcessorFactoryTest, TestExtendedDismaxParser, SuggesterTSTTest, SuggesterTest, TestStressLucene, TestTrie, PolyFieldTest, NoCacheHeaderTest, WordBreakSolrSpellCheckerTest, SchemaVersionSpecificBehaviorTest, TestPseudoReturnFields, TestUpdate, TestWordDelimiterFilterFactory, QueryEqualityTest, DefaultValueUpdateProcessorTest, StatelessScriptUpdateProcessorFactoryTest, SortByFunctionTest, XsltUpdateRequestHandlerTest, IndexBasedSpellCheckerTest, DisMaxRequestHandlerTest, TestWriterPerf, DirectSolrSpellCheckerTest, DocumentAnalysisRequestHandlerTest, TestQueryTypes, TestOmitPositions, FileBasedSpellCheckerTest, TermsComponentTest, XmlUpdateRequestHandlerTest, DocumentBuilderTest, TestValueSourceCache, TermVectorComponentTest, RequiredFieldsTest, TestArbitraryIndexDir, FastVectorHighlighterTest, IndexSchemaRuntimeFieldTest, LoggingHandlerTest, RegexBoostProcessorTest, SolrPluginUtilsTest, ReturnFieldsTest, JSONWriterTest, TestCSVResponseWriter, MBeansHandlerTest, TestAnalyzedSuggestions, UniqFieldsUpdateProcessorFactoryTest, TestPHPSerializedResponseWriter, HighlighterConfigTest, SOLR749Test, UpdateParamsTest, TestSolrIndexConfig, SolrIndexConfigTest, TestStressRecovery, MultiTermTest, TestBinaryField, TestSearchPerf, NumericFieldsTest, OutputWriterTest, TestFuzzyAnalyzedSuggestions, TestCharFilters, TestSweetSpotSimilarityFactory, TestDFRSimilarityFactory, TestLMDirichletSimilarityFactory, TestFastWriter, TestFastOutputStream, URLClassifyProcessorTest, DateMathParserTest, PrimUtilsTest, SpellingQueryConverterTest, ClusterStateTest, TestLRUCache, TestDocumentBuilder, SystemInfoHandlerTest, FileUtilsTest, DistributedMLTComponentTest, TestCrossCoreJoin, TestCursorMarkWithoutUniqueKey, TestSimpleTrackingShardHandler, TestTolerantSearch, TestEmbeddedSolrServerConstructors, ActionThrottleTest, AsyncCallRequestStatusResponseTest, CleanupOldIndexTest, CollectionStateFormat2Test, DeleteInactiveReplicaTest, DistribDocExpirationUpdateProcessorTest, LeaderInitiatedRecoveryOnShardRestartTest]
   [junit4] Completed [439/542] on J2 in 364.94s, 1 test, 1 error <<< FAILURES!

[...truncated 365 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 542 suites (7 ignored), 2143 tests, 1 error, 31 ignored (18 assumptions) [seed: 7F78F76DDF75FAD1]

Total time: 198 minutes 25 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 992 - Still Failing

[Apache Jenkins Server <je...@builds.apache.org>]

Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/992/

2 tests failed.
FAILED:  org.apache.lucene.codecs.memory.TestDirectDocValuesFormat.testRandomSortedBytes

Error Message:
Test abandoned because suite timeout was reached.

Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
	at __randomizedtesting.SeedInfo.seed([A183E624F885474E]:0)


FAILED:  junit.framework.TestSuite.org.apache.lucene.codecs.memory.TestDirectDocValuesFormat

Error Message:
Suite timeout exceeded (>= 7200000 msec).

Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
	at __randomizedtesting.SeedInfo.seed([A183E624F885474E]:0)




Build Log:
[...truncated 6374 lines...]
   [junit4] Suite: org.apache.lucene.codecs.memory.TestDirectDocValuesFormat
   [junit4]   2> Oct 21, 2015 2:34:09 AM com.carrotsearch.randomizedtesting.ThreadLeakControl$2 evaluate
   [junit4]   2> WARNING: Suite execution timed out: org.apache.lucene.codecs.memory.TestDirectDocValuesFormat
   [junit4]   2>    1) Thread[id=2756, name=SUITE-TestDirectDocValuesFormat-seed#[A183E624F885474E], state=RUNNABLE, group=TGRP-TestDirectDocValuesFormat]
   [junit4]   2>         at java.lang.Thread.getStackTrace(Thread.java:1589)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$4.run(ThreadLeakControl.java:688)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$4.run(ThreadLeakControl.java:685)
   [junit4]   2>         at java.security.AccessController.doPrivileged(Native Method)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl.getStackTrace(ThreadLeakControl.java:685)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl.getThreadsWithTraces(ThreadLeakControl.java:701)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl.formatThreadStacksFull(ThreadLeakControl.java:681)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl.access$1000(ThreadLeakControl.java:64)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$2.evaluate(ThreadLeakControl.java:414)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:676)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner.access$200(RandomizedRunner.java:140)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$2.run(RandomizedRunner.java:587)
   [junit4]   2>    2) Thread[id=1, name=main, state=WAITING, group=main]
   [junit4]   2>         at java.lang.Object.wait(Native Method)
   [junit4]   2>         at java.lang.Thread.join(Thread.java:1281)
   [junit4]   2>         at java.lang.Thread.join(Thread.java:1355)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:597)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner.run(RandomizedRunner.java:446)
   [junit4]   2>         at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.execute(SlaveMain.java:244)
   [junit4]   2>         at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.main(SlaveMain.java:355)
   [junit4]   2>         at com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe.main(SlaveMainSafe.java:10)
   [junit4]   2>    3) Thread[id=8, name=JUnit4-serializer-daemon, state=TIMED_WAITING, group=main]
   [junit4]   2>         at java.lang.Thread.sleep(Native Method)
   [junit4]   2>         at com.carrotsearch.ant.tasks.junit4.events.Serializer$1.run(Serializer.java:49)
   [junit4]   2>    4) Thread[id=2757, name=TEST-TestDirectDocValuesFormat.testRandomSortedBytes-seed#[A183E624F885474E], state=TIMED_WAITING, group=TGRP-TestDirectDocValuesFormat]
   [junit4]   2>         at java.lang.Object.wait(Native Method)
   [junit4]   2>         at org.apache.lucene.index.IndexWriter.doWait(IndexWriter.java:4296)
   [junit4]   2>         at org.apache.lucene.index.IndexWriter.forceMerge(IndexWriter.java:1785)
   [junit4]   2>         at org.apache.lucene.index.IndexWriter.forceMerge(IndexWriter.java:1719)
   [junit4]   2>         at org.apache.lucene.index.RandomIndexWriter.doRandomForceMerge(RandomIndexWriter.java:340)
   [junit4]   2>         at org.apache.lucene.index.RandomIndexWriter.getReader(RandomIndexWriter.java:355)
   [junit4]   2>         at org.apache.lucene.index.RandomIndexWriter.getReader(RandomIndexWriter.java:301)
   [junit4]   2>         at org.apache.lucene.index.BaseDocValuesFormatTestCase.testRandomSortedBytes(BaseDocValuesFormatTestCase.java:1146)
   [junit4]   2>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   2>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   [junit4]   2>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2>         at java.lang.reflect.Method.invoke(Method.java:606)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1665)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:864)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:900)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:914)
   [junit4]   2>         at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
   [junit4]   2>         at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2>         at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
   [junit4]   2>         at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2>         at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:873)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:775)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:809)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:820)
   [junit4]   2>         at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   2>         at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2>         at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2>         at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
   [junit4]   2>         at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestDirectDocValuesFormat -Dtests.method=testRandomSortedBytes -Dtests.seed=A183E624F885474E -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=en_MT -Dtests.timezone=Europe/San_Marino -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
   [junit4] ERROR   7191s J0 | TestDirectDocValuesFormat.testRandomSortedBytes <<<
   [junit4]    > Throwable #1: java.lang.Exception: Test abandoned because suite timeout was reached.
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([A183E624F885474E]:0)
   [junit4]   2> Oct 21, 2015 2:34:09 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
   [junit4]   2> Oct 21, 2015 2:34:29 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> SEVERE: 1 thread leaked from SUITE scope at org.apache.lucene.codecs.memory.TestDirectDocValuesFormat: 
   [junit4]   2>    1) Thread[id=2757, name=TEST-TestDirectDocValuesFormat.testRandomSortedBytes-seed#[A183E624F885474E], state=TIMED_WAITING, group=TGRP-TestDirectDocValuesFormat]
   [junit4]   2>         at java.lang.Object.wait(Native Method)
   [junit4]   2>         at org.apache.lucene.index.IndexWriter.doWait(IndexWriter.java:4296)
   [junit4]   2>         at org.apache.lucene.index.IndexWriter.forceMerge(IndexWriter.java:1785)
   [junit4]   2>         at org.apache.lucene.index.IndexWriter.forceMerge(IndexWriter.java:1719)
   [junit4]   2>         at org.apache.lucene.index.RandomIndexWriter.doRandomForceMerge(RandomIndexWriter.java:340)
   [junit4]   2>         at org.apache.lucene.index.RandomIndexWriter.getReader(RandomIndexWriter.java:355)
   [junit4]   2>         at org.apache.lucene.index.RandomIndexWriter.getReader(RandomIndexWriter.java:301)
   [junit4]   2>         at org.apache.lucene.index.BaseDocValuesFormatTestCase.testRandomSortedBytes(BaseDocValuesFormatTestCase.java:1146)
   [junit4]   2>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   2>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   [junit4]   2>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2>         at java.lang.reflect.Method.invoke(Method.java:606)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1665)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:864)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:900)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:914)
   [junit4]   2>         at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
   [junit4]   2>         at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2>         at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
   [junit4]   2>         at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2>         at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:873)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:775)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:809)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:820)
   [junit4]   2>         at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   2>         at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2>         at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2>         at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
   [junit4]   2>         at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> Oct 21, 2015 2:34:29 AM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
   [junit4]   2> INFO: Starting to interrupt leaked threads:
   [junit4]   2>    1) Thread[id=2757, name=TEST-TestDirectDocValuesFormat.testRandomSortedBytes-seed#[A183E624F885474E], state=TIMED_WAITING, group=TGRP-TestDirectDocValuesFormat]
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build/codecs/test/J0/temp/lucene.codecs.memory.TestDirectDocValuesFormat_A183E624F885474E-001
   [junit4]   2> Oct 20, 2015 8:34:29 PM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
   [junit4]   2> INFO: All leaked threads terminated.
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene53): {}, docValues:{}, sim=RandomSimilarityProvider(queryNorm=false,coord=yes): {id=DFR I(n)L3(800.0), docId=DFR I(F)L3(800.0)}, locale=en_MT, timezone=Europe/San_Marino
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_72 (64-bit)/cpus=4,threads=1,free=211788168,total=341311488
   [junit4]   2> NOTE: All tests run in this JVM: [TestVarGapFixedIntervalPostingsFormat, TestAutoPrefixPostingsFormat, TestSimpleTextNormsFormat, TestMemoryPostingsFormat, TestSimpleTextPostingsFormat, TestDirectDocValuesFormat]
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestDirectDocValuesFormat -Dtests.seed=A183E624F885474E -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=en_MT -Dtests.timezone=Europe/San_Marino -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
   [junit4] ERROR   0.00s J0 | TestDirectDocValuesFormat (suite) <<<
   [junit4]    > Throwable #1: java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([A183E624F885474E]:0)
   [junit4] Completed [21/21] on J0 in 7220.16s, 9 tests, 2 errors <<< FAILURES!

[...truncated 12 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:792: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/build.xml:471: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:2248: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/module-build.xml:58: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1006: There were test failures: 21 suites, 616 tests, 1 suite-level error, 1 error, 1 ignored (1 assumption) [seed: A183E624F885474E]

Total time: 215 minutes 57 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any