You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Policeman Jenkins Server <je...@thetaphi.de> on 2015/05/22 09:01:30 UTC

[JENKINS] Lucene-Solr-trunk-Windows (32bit/jdk1.8.0_45) - Build # 4840 - Still Failing!

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Windows/4840/
Java: 32bit/jdk1.8.0_45 -server -XX:+UseG1GC

2 tests failed.
FAILED:  org.apache.solr.search.TestSurroundQueryParser.testQueryParser

Error Message:
Test abandoned because suite timeout was reached.

Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
	at __randomizedtesting.SeedInfo.seed([ABEA39500764A10D]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.search.TestSurroundQueryParser

Error Message:
Suite timeout exceeded (>= 7200000 msec).

Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
	at __randomizedtesting.SeedInfo.seed([ABEA39500764A10D]:0)




Build Log:
[...truncated 10968 lines...]
   [junit4] Suite: org.apache.solr.search.TestSurroundQueryParser
   [junit4]   2> Creating dataDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.search.TestSurroundQueryParser ABEA39500764A10D-001\init-core-data-001
   [junit4]   2> 1670859 T9932 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (true) and clientAuth (false)
   [junit4]   2> 1670861 T9932 oas.SolrTestCaseJ4.initCore ####initCore
   [junit4]   2> 1670861 T9932 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\'
   [junit4]   2> 1670863 T9932 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/C:/Users/JenkinsSlave/workspace/Lucene-Solr-trunk-Windows/solr/core/src/test-files/solr/collection1/lib/classes/' to classloader
   [junit4]   2> 1670863 T9932 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/C:/Users/JenkinsSlave/workspace/Lucene-Solr-trunk-Windows/solr/core/src/test-files/solr/collection1/lib/README' to classloader
   [junit4]   2> 1670905 T9932 oasc.SolrConfig.refreshRequestParams current version of requestparams : -1
   [junit4]   2> 1670914 T9932 oasc.SolrConfig.<init> Using Lucene MatchVersion: 6.0.0
   [junit4]   2> 1670930 T9932 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1670931 T9932 oass.IndexSchema.readSchema Reading Solr Schema from C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\schemasurround.xml
   [junit4]   2> 1670936 T9932 oass.IndexSchema.readSchema [null] Schema name=test
   [junit4]   2> 1671024 T9932 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 1671025 T9932 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 1671025 T9932 oass.IndexSchema.loadCopyFields WARN Field text is not multivalued and destination for multiple copyFields (2)
   [junit4]   2> 1671032 T9932 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1671032 T9932 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr
   [junit4]   2> 1671032 T9932 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\'
   [junit4]   2> 1671049 T9932 oasc.CoreContainer.<init> New CoreContainer 8681450
   [junit4]   2> 1671049 T9932 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\]
   [junit4]   2> 1671049 T9932 oasc.CoreContainer.load loading shared library: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\lib
   [junit4]   2> 1671050 T9932 oasc.SolrResourceLoader.addToClassLoader WARN Can't find (or read) directory to add to classloader: lib (resolved as: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\lib).
   [junit4]   2> 1671058 T9932 oashc.HttpShardHandlerFactory.init created with socketTimeout : 600000,connTimeout : 60000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1671065 T9932 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=30000&connTimeout=30000&retry=true
   [junit4]   2> 1671070 T9932 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1671071 T9932 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1671071 T9932 oasc.CoreContainer.load Node Name: testNode
   [junit4]   2> 1671071 T9932 oasc.CoreContainer.initializeAuthenticationPlugin No authentication plugin used.
   [junit4]   2> 1671072 T9932 oasc.CoreDescriptor.<init> CORE DESCRIPTOR: {name=collection1, config=solrconfig.xml, transient=false, schema=schemasurround.xml, loadOnStartup=true, instanceDir=collection1, collection=collection1, absoluteInstDir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\, dataDir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.search.TestSurroundQueryParser ABEA39500764A10D-001\init-core-data-001, shard=shard1}
   [junit4]   2> 1671074 T9933 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\'
   [junit4]   2> 1671074 T9933 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/C:/Users/JenkinsSlave/workspace/Lucene-Solr-trunk-Windows/solr/core/src/test-files/solr/collection1/lib/classes/' to classloader
   [junit4]   2> 1671075 T9933 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/C:/Users/JenkinsSlave/workspace/Lucene-Solr-trunk-Windows/solr/core/src/test-files/solr/collection1/lib/README' to classloader
   [junit4]   2> 1671095 T9933 oasc.SolrConfig.refreshRequestParams current version of requestparams : -1
   [junit4]   2> 1671102 T9933 oasc.SolrConfig.<init> Using Lucene MatchVersion: 6.0.0
   [junit4]   2> 1671120 T9933 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1671121 T9933 oass.IndexSchema.readSchema Reading Solr Schema from C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\schemasurround.xml
   [junit4]   2> 1671125 T9933 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 1671198 T9933 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 1671199 T9933 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 1671199 T9933 oass.IndexSchema.loadCopyFields WARN Field text is not multivalued and destination for multiple copyFields (2)
   [junit4]   2> 1671205 T9933 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from instancedir C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\
   [junit4]   2> 1671205 T9933 x:collection1 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 1671206 T9933 x:collection1 oasc.SolrCore.<init> [[collection1] ] Opening new SolrCore at [C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\], dataDir=[null]
   [junit4]   2> 1671206 T9933 x:collection1 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1217b81
   [junit4]   2> 1671207 T9933 x:collection1 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.search.TestSurroundQueryParser ABEA39500764A10D-001\init-core-data-001\
   [junit4]   2> 1671207 T9933 x:collection1 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.search.TestSurroundQueryParser ABEA39500764A10D-001\init-core-data-001\index/
   [junit4]   2> 1671207 T9933 x:collection1 oasc.SolrCore.initIndex WARN [collection1] Solr index directory 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.search.TestSurroundQueryParser ABEA39500764A10D-001\init-core-data-001\index' doesn't exist. Creating new index...
   [junit4]   2> 1671207 T9933 x:collection1 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.search.TestSurroundQueryParser ABEA39500764A10D-001\init-core-data-001\index
   [junit4]   2> 1671207 T9933 x:collection1 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=21, maxMergeAtOnceExplicit=46, maxMergedSegmentMB=77.3603515625, floorSegmentMB=1.158203125, forceMergeDeletesPctAllowed=18.706457216408182, segmentsPerTier=12.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.6779995195324177
   [junit4]   2> 1671209 T9933 x:collection1 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@743283 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@9e9a43),segFN=segments_1,generation=1}
   [junit4]   2> 1671209 T9933 x:collection1 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 1671237 T9933 x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 1671238 T9933 x:collection1 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 1671238 T9933 x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe-allfields"
   [junit4]   2> 1671238 T9933 x:collection1 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
   [junit4]   2> 1671238 T9933 x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1671238 T9933 x:collection1 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1671239 T9933 x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "uniq-fields"
   [junit4]   2> 1671239 T9933 x:collection1 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
   [junit4]   2> 1671239 T9933 x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 1671239 T9933 x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1671239 T9933 x:collection1 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1671240 T9933 x:collection1 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 1671244 T9933 x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 1671245 T9933 x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 1671247 T9933 x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 1671248 T9933 x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 1671251 T9933 x:collection1 oasc.PluginBag.createPlugin requestHandler : 'lazy' created with startup=lazy 
   [junit4]   2> 1671254 T9933 x:collection1 oasc.RequestHandlers.initHandlersFromConfig Registered paths: /admin/mbeans,standard,spellCheckCompRH,lazy,/update/json/docs,/admin/luke,spellCheckWithWordbreak_Direct,tvrh,/get,/admin/properties,/update/json,mltrh,/admin/threads,/search-facet-def,/update/csv,/search-facet-invariants,/admin/segments,spellCheckCompRH1,/mlt,/admin/system,/replication,dismax,defaults,/config,/schema,/admin/plugins,/admin/logging,spellCheckWithWordbreak,mock,/admin/ping,/update,/admin/file,/terms,spellCheckCompRH_Direct,/debug/dump
   [junit4]   2> 1671254 T9933 x:collection1 oasc.SolrCore.initDeprecatedSupport WARN solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
   [junit4]   2> 1671254 T9933 x:collection1 oasc.SolrCore.initStatsCache Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 1671255 T9933 x:collection1 oasu.UpdateHandler.<init> Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1671256 T9933 x:collection1 oasu.UpdateLog.init Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=256
   [junit4]   2> 1671256 T9933 x:collection1 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 1671256 T9933 x:collection1 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 1671256 T9933 x:collection1 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=11, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.3821840231135786]
   [junit4]   2> 1671257 T9933 x:collection1 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@743283 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@9e9a43),segFN=segments_1,generation=1}
   [junit4]   2> 1671257 T9933 x:collection1 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 1671257 T9933 x:collection1 oass.SolrIndexSearcher.<init> Opening Searcher@c48624[collection1] main
   [junit4]   2> 1671257 T9933 x:collection1 oasu.UpdateLog.onFirstSearcher On first searcher opened, looking up max value of version field
   [junit4]   2> 1671258 T9933 x:collection1 oasu.VersionInfo.getMaxVersionFromIndex Refreshing highest value of _version_ for 256 version buckets from index
   [junit4]   2> 1671258 T9933 x:collection1 oasu.VersionInfo.getMaxVersionFromIndex WARN No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 1671258 T9933 x:collection1 oasu.UpdateLog.seedBucketsWithHighestVersion WARN Could not find max version in index or recent updates, using new clock 1501844820413382656
   [junit4]   2> 1671258 T9933 x:collection1 oasu.UpdateLog.seedBucketsWithHighestVersion Took 0 ms to seed version buckets with highest version 1501844820413382656
   [junit4]   2> 1671258 T9933 x:collection1 oasr.RestManager.init Initializing RestManager with initArgs: {storageIO=org.apache.solr.rest.ManagedResourceStorage$InMemoryStorageIO}
   [junit4]   2> 1671258 T9933 x:collection1 oasr.ManagedResourceStorage.load Reading _rest_managed.json using InMemoryStorage
   [junit4]   2> 1671259 T9933 x:collection1 oasr.RestManager.init Initializing 0 registered ManagedResources
   [junit4]   2> 1671259 T9933 x:collection1 oashc.SpellCheckComponent.inform Initializing spell checkers
   [junit4]   2> 1671266 T9933 x:collection1 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
   [junit4]   2> 1671313 T9933 x:collection1 oash.ReplicationHandler.inform Commits will be reserved for  10000
   [junit4]   2> 1671314 T9933 x:collection1 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 1671314 T9934 x:collection1 oashc.SpellCheckComponent$SpellCheckerListener.newSearcher Loading spell index for spellchecker: default
   [junit4]   2> 1671314 T9934 x:collection1 oashc.SpellCheckComponent$SpellCheckerListener.newSearcher Loading spell index for spellchecker: direct
   [junit4]   2> 1671315 T9934 x:collection1 oashc.SpellCheckComponent$SpellCheckerListener.newSearcher Loading spell index for spellchecker: wordbreak
   [junit4]   2> 1671315 T9934 x:collection1 oashc.SpellCheckComponent$SpellCheckerListener.newSearcher Loading spell index for spellchecker: multipleFields
   [junit4]   2> 1671315 T9934 x:collection1 oashc.SpellCheckComponent$SpellCheckerListener.newSearcher Loading spell index for spellchecker: jarowinkler
   [junit4]   2> 1671316 T9934 x:collection1 oashc.SpellCheckComponent$SpellCheckerListener.newSearcher Loading spell index for spellchecker: external
   [junit4]   2> 1671316 T9934 x:collection1 oashc.SpellCheckComponent$SpellCheckerListener.newSearcher Loading spell index for spellchecker: freq
   [junit4]   2> 1671316 T9934 x:collection1 oashc.SpellCheckComponent$SpellCheckerListener.newSearcher Loading spell index for spellchecker: fqcn
   [junit4]   2> 1671317 T9932 oas.SolrTestCaseJ4.initCore ####initCore end
   [junit4]   2> 1671321 T9932 oas.SolrTestCaseJ4.setUp ###Starting testQueryParser
   [junit4]   2> ASYNC  NEW_CORE C2968 name=collection1 org.apache.solr.core.SolrCore@162ef45
   [junit4]   2> 1671326 T9932 C2968 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[1 (1501844820479442944)]} 0 5
   [junit4]   2> 1671329 T9932 C2968 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[2 (1501844820486782976)]} 0 1
   [junit4]   2> 1671330 T9932 C2968 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[3 (1501844820487831552)]} 0 1
   [junit4]   2> 1671330 T9932 C2968 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 1671334 T9932 C2968 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@743283 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@9e9a43),segFN=segments_1,generation=1}
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@743283 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@9e9a43),segFN=segments_2,generation=2}
   [junit4]   2> 1671336 T9932 C2968 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
   [junit4]   2> 1671337 T9932 C2968 oasc.SolrCore.getSearcher WARN [collection1] PERFORMANCE WARNING: Overlapping onDeckSearchers=2
   [junit4]   2> 1671338 T9932 C2968 oass.SolrIndexSearcher.<init> Opening Searcher@1fd3c24[collection1] main
   [junit4]   2> 1671339 T9932 C2968 oasu.UpdateLog.onFirstSearcher On first searcher opened, looking up max value of version field
   [junit4]   2> 1671317 T9934 x:collection1 oashc.SpellCheckComponent$SpellCheckerListener.newSearcher Loading spell index for spellchecker: perDict
   [junit4]   2> 1671340 T9934 x:collection1 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@c48624[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 7222771 T9931 ccr.ThreadLeakControl$2.evaluate WARNING Suite execution timed out: org.apache.solr.search.TestSurroundQueryParser
   [junit4]   2> 	==== jstack at approximately timeout time ====
   [junit4]   2> 	"RMI TCP Connection(idle)" ID=9959 TIMED_WAITING on java.util.concurrent.SynchronousQueue$TransferStack@861cab
   [junit4]   2> 		at sun.misc.Unsafe.park(Native Method)
   [junit4]   2> 		- timed waiting on java.util.concurrent.SynchronousQueue$TransferStack@861cab
   [junit4]   2> 		at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]   2> 		at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
   [junit4]   2> 		at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
   [junit4]   2> 		at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
   [junit4]   2> 		at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
   [junit4]   2> 		at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]   2> 		at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]   2> 		at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 	
   [junit4]   2> 	"Thread-2836" ID=9935 WAITING on java.lang.Object@5a8721
   [junit4]   2> 		at java.lang.Object.wait(Native Method)
   [junit4]   2> 		- waiting on java.lang.Object@5a8721
   [junit4]   2> 		at java.lang.Object.wait(Object.java:502)
   [junit4]   2> 		at org.apache.solr.core.CloserThread.run(CoreContainer.java:1035)
   [junit4]   2> 	
   [junit4]   2> 	"searcherExecutor-4207-thread-1" ID=9934 WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@12a1b2c
   [junit4]   2> 		at sun.misc.Unsafe.park(Native Method)
   [junit4]   2> 		- waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@12a1b2c
   [junit4]   2> 		at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
   [junit4]   2> 		at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
   [junit4]   2> 		at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]   2> 		at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
   [junit4]   2> 		at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]   2> 		at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]   2> 		at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 	
   [junit4]   2> 	"TEST-TestSurroundQueryParser.testQueryParser-seed#[ABEA39500764A10D]" ID=9932 WAITING on java.util.concurrent.locks.ReentrantReadWriteLock$FairSync@1c2988e
   [junit4]   2> 		at sun.misc.Unsafe.park(Native Method)
   [junit4]   2> 		- waiting on java.util.concurrent.locks.ReentrantReadWriteLock$FairSync@1c2988e
   [junit4]   2> 		at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
   [junit4]   2> 		at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
   [junit4]   2> 		at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870)
   [junit4]   2> 		at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199)
   [junit4]   2> 		at java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.lock(ReentrantReadWriteLock.java:943)
   [junit4]   2> 		at org.apache.solr.update.VersionInfo.blockUpdates(VersionInfo.java:119)
   [junit4]   2> 		at org.apache.solr.update.UpdateLog.onFirstSearcher(UpdateLog.java:1604)
   [junit4]   2> 		at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1810)
   [junit4]   2> 		at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1505)
   [junit4]   2> 		at org.apache.solr.update.DirectUpdateHandler2.commit(DirectUpdateHandler2.java:617)
   [junit4]   2> 		- locked java.lang.Object@1cde08
   [junit4]   2> 		at org.apache.solr.update.processor.RunUpdateProcessor.processCommit(RunUpdateProcessorFactory.java:95)
   [junit4]   2> 		at org.apache.solr.update.processor.UpdateRequestProcessor.processCommit(UpdateRequestProcessor.java:64)
   [junit4]   2> 		at org.apache.solr.update.processor.DistributedUpdateProcessor.doLocalCommit(DistributedUpdateProcessor.java:1635)
   [junit4]   2> 		at org.apache.solr.update.processor.DistributedUpdateProcessor.processCommit(DistributedUpdateProcessor.java:1612)
   [junit4]   2> 		at org.apache.solr.update.processor.LogUpdateProcessor.processCommit(LogUpdateProcessorFactory.java:161)
   [junit4]   2> 		at org.apache.solr.handler.loader.XMLLoader.processUpdate(XMLLoader.java:270)
   [junit4]   2> 		at org.apache.solr.handler.loader.XMLLoader.load(XMLLoader.java:177)
   [junit4]   2> 		at org.apache.solr.handler.UpdateRequestHandler$1.load(UpdateRequestHandler.java:98)
   [junit4]   2> 		at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:74)
   [junit4]   2> 		at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:143)
   [junit4]   2> 		at org.apache.solr.core.SolrCore.execute(SolrCore.java:2051)
   [junit4]   2> 		at org.apache.solr.servlet.DirectSolrConnection.request(DirectSolrConnection.java:131)
   [junit4]   2> 		at org.apache.solr.util.TestHarness.update(TestHarness.java:267)
   [junit4]   2> 		at org.apache.solr.util.BaseTestHarness.checkUpdateStatus(BaseTestHarness.java:282)
   [junit4]   2> 		at org.apache.solr.util.BaseTestHarness.validateUpdate(BaseTestHarness.java:252)
   [junit4]   2> 		at org.apache.solr.SolrTestCaseJ4.checkUpdateU(SolrTestCaseJ4.java:748)
   [junit4]   2> 		at org.apache.solr.SolrTestCaseJ4.assertU(SolrTestCaseJ4.java:727)
   [junit4]   2> 		at org.apache.solr.SolrTestCaseJ4.assertU(SolrTestCaseJ4.java:721)
   [junit4]   2> 		at org.apache.solr.search.TestSurroundQueryParser.testQueryParser(TestSurroundQueryParser.java:54)
   [junit4]   2> 		at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   2> 		at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   [junit4]   2> 		at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2> 		at java.lang.reflect.Method.invoke(Method.java:497)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2> 		at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
   [junit4]   2> 		at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2> 		at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
   [junit4]   2> 		at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2> 		at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2> 		at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 		at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 		at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   2> 		at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2> 		at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2> 		at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   2> 		at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 		Locked synchronizers:
   [junit4]   2> 		- java.util.concurrent.locks.ReentrantLock$NonfairSync@a9e3e4
   [junit4]   2> 		- java.util.concurrent.locks.ReentrantLock$FairSync@360f9f
   [junit4]   2> 	
   [junit4]   2> 	"SUITE-TestSurroundQueryParser-seed#[ABEA39500764A10D]" ID=9931 RUNNABLE
   [junit4]   2> 		at sun.management.ThreadImpl.dumpThreads0(Native Method)
   [junit4]   2> 		at sun.management.ThreadImpl.dumpAllThreads(ThreadImpl.java:446)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.ThreadLeakControl.formatThreadStacksFull(ThreadLeakControl.java:671)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.ThreadLeakControl.access$900(ThreadLeakControl.java:62)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.ThreadLeakControl$2.evaluate(ThreadLeakControl.java:412)
   [junit4]   2> 		- locked java.lang.Object@118f5a0
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:651)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.RandomizedRunner.access$200(RandomizedRunner.java:138)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.RandomizedRunner$1.run(RandomizedRunner.java:568)
   [junit4]   2> 	
   [junit4]   2> 	"RMI TCP Accept-0" ID=9595 RUNNABLE (in native code)
   [junit4]   2> 		at java.net.DualStackPlainSocketImpl.accept0(Native Method)
   [junit4]   2> 		at java.net.DualStackPlainSocketImpl.socketAccept(DualStackPlainSocketImpl.java:131)
   [junit4]   2> 		at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:404)
   [junit4]   2> 		at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:199)
   [junit4]   2> 		- locked java.net.SocksSocketImpl@e799e4
   [junit4]   2> 		at java.net.ServerSocket.implAccept(ServerSocket.java:545)
   [junit4]   2> 		at java.net.ServerSocket.accept(ServerSocket.java:513)
   [junit4]   2> 		at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.executeAcceptLoop(TCPTransport.java:400)
   [junit4]   2> 		at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.run(TCPTransport.java:372)
   [junit4]   2> 		at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 	
   [junit4]   2> 	"RMI RenewClean-[127.0.0.1:54234]" ID=9591 TIMED_WAITING on java.lang.ref.ReferenceQueue$Lock@e734d0
   [junit4]   2> 		at java.lang.Object.wait(Native Method)
   [junit4]   2> 		- timed waiting on java.lang.ref.ReferenceQueue$Lock@e734d0
   [junit4]   2> 		at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:143)
   [junit4]   2> 		at sun.rmi.transport.DGCClient$EndpointEntry$RenewCleanThread.run(DGCClient.java:536)
   [junit4]   2> 		at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 	
   [junit4]   2> 	"RMI Scheduler(0)" ID=9590 TIMED_WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@108c3aa
   [junit4]   2> 		at sun.misc.Unsafe.park(Native Method)
   [junit4]   2> 		- timed waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@108c3aa
   [junit4]   2> 		at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]   2> 		at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
   [junit4]   2> 		at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
   [junit4]   2> 		at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
   [junit4]   2> 		at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
   [junit4]   2> 		at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]   2> 		at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]   2> 		at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 	
   [junit4]   2> 	"GC Daemon" ID=9588 TIMED_WAITING on sun.misc.GC$LatencyLock@1b187de
   [junit4]   2> 		at java.lang.Object.wait(Native Method)
   [junit4]   2> 		- timed waiting on sun.misc.GC$LatencyLock@1b187de
   [junit4]   2> 		at sun.misc.GC$Daemon.run(GC.java:117)
   [junit4]   2> 	
   [junit4]   2> 	"RMI Reaper" ID=9587 WAITING on java.lang.ref.ReferenceQueue$Lock@1271335
   [junit4]   2> 		at java.lang.Object.wait(Native Method)
   [junit4]   2> 		- waiting on java.lang.ref.ReferenceQueue$Lock@1271335
   [junit4]   2> 		at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:143)
   [junit4]   2> 		at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:164)
   [junit4]   2> 		at sun.rmi.transport.ObjectTable$Reaper.run(ObjectTable.java:351)
   [junit4]   2> 		at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 	
   [junit4]   2> 	"RMI TCP Accept-0" ID=9586 RUNNABLE (in native code)
   [junit4]   2> 		at java.net.DualStackPlainSocketImpl.accept0(Native Method)
   [junit4]   2> 		at java.net.DualStackPlainSocketImpl.socketAccept(DualStackPlainSocketImpl.java:131)
   [junit4]   2> 		at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:404)
   [junit4]   2> 		at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:199)
   [junit4]   2> 		- locked java.net.SocksSocketImpl@97f435
   [junit4]   2> 		at java.net.ServerSocket.implAccept(ServerSocket.java:545)
   [junit4]   2> 		at java.net.ServerSocket.accept(ServerSocket.java:513)
   [junit4]   2> 		at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.executeAcceptLoop(TCPTransport.java:400)
   [junit4]   2> 		at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.run(TCPTransport.java:372)
   [junit4]   2> 		at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 	
   [junit4]   2> 	"RMI TCP Accept-0" ID=9585 RUNNABLE (in native code)
   [junit4]   2> 		at java.net.DualStackPlainSocketImpl.accept0(Native Method)
   [junit4]   2> 		at java.net.DualStackPlainSocketImpl.socketAccept(DualStackPlainSocketImpl.java:131)
   [junit4]   2> 		at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:404)
   [junit4]   2> 		at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:199)
   [junit4]   2> 		- locked java.net.SocksSocketImpl@1e57e5e
   [junit4]   2> 		at java.net.ServerSocket.implAccept(ServerSocket.java:545)
   [junit4]   2> 		at java.net.ServerSocket.accept(ServerSocket.java:513)
   [junit4]   2> 		at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.executeAcceptLoop(TCPTransport.java:400)
   [junit4]   2> 		at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.run(TCPTransport.java:372)
   [junit4]   2> 		at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 	
   [junit4]   2> 	"TimeLimitedCollector timer thread" ID=907 TIMED_WAITING
   [junit4]   2> 		at java.lang.Thread.sleep(Native Method)
   [junit4]   2> 		at org.apache.lucene.search.TimeLimitingCollector$TimerThread.run(TimeLimitingCollector.java:256)
   [junit4]   2> 	
   [junit4]   2> 	"JUnit4-serializer-daemon" ID=11 TIMED_WAITING
   [junit4]   2> 		at java.lang.Thread.sleep(Native Method)
   [junit4]   2> 		at com.carrotsearch.ant.tasks.junit4.events.Serializer$1.run(Serializer.java:47)
   [junit4]   2> 	
   [junit4]   2> 	"Attach Listener" ID=6 RUNNABLE
   [junit4]   2> 	
   [junit4]   2> 	"Signal Dispatcher" ID=5 RUNNABLE
   [junit4]   2> 	
   [junit4]   2> 	"Finalizer" ID=3 WAITING on java.lang.ref.ReferenceQueue$Lock@cb864b
   [junit4]   2> 		at java.lang.Object.wait(Native Method)
   [junit4]   2> 		- waiting on java.lang.ref.ReferenceQueue$Lock@cb864b
   [junit4]   2> 		at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:143)
   [junit4]   2> 		at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:164)
   [junit4]   2> 		at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:209)
   [junit4]   2> 	
   [junit4]   2> 	"Reference Handler" ID=2 WAITING on java.lang.ref.Reference$Lock@1de975c
   [junit4]   2> 		at java.lang.Object.wait(Native Method)
   [junit4]   2> 		- waiting on java.lang.ref.Reference$Lock@1de975c
   [junit4]   2> 		at java.lang.Object.wait(Object.java:502)
   [junit4]   2> 		at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:157)
   [junit4]   2> 	
   [junit4]   2> 	"main" ID=1 WAITING on com.carrotsearch.randomizedtesting.RandomizedRunner$1@1bb1690
   [junit4]   2> 		at java.lang.Object.wait(Native Method)
   [junit4]   2> 		- waiting on com.carrotsearch.randomizedtesting.RandomizedRunner$1@1bb1690
   [junit4]   2> 		at java.lang.Thread.join(Thread.java:1245)
   [junit4]   2> 		at java.lang.Thread.join(Thread.java:1319)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:578)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.RandomizedRunner.run(RandomizedRunner.java:444)
   [junit4]   2> 		at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.execute(SlaveMain.java:199)
   [junit4]   2> 		at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.main(SlaveMain.java:310)
   [junit4]   2> 		at com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe.main(SlaveMainSafe.java:12)
   [junit4]   2> 	
   [junit4]   2> 	^^==============================================
   [junit4]   2> 	
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestSurroundQueryParser -Dtests.method=testQueryParser -Dtests.seed=ABEA39500764A10D -Dtests.slow=true -Dtests.locale=es_US -Dtests.timezone=Atlantic/Azores -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
   [junit4] ERROR   7222s J0 | TestSurroundQueryParser.testQueryParser <<<
   [junit4]    > Throwable #1: java.lang.Exception: Test abandoned because suite timeout was reached.
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([ABEA39500764A10D]:0)
   [junit4]   2> 7222778 T9931 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 3 leaked thread(s).
   [junit4]   2> 7242920 T9931 ccr.ThreadLeakControl.checkThreadLeaks SEVERE 3 threads leaked from SUITE scope at org.apache.solr.search.TestSurroundQueryParser: 
   [junit4]   2> 	   1) Thread[id=9935, name=Thread-2836, state=WAITING, group=TGRP-TestSurroundQueryParser]
   [junit4]   2> 	        at java.lang.Object.wait(Native Method)
   [junit4]   2> 	        at java.lang.Object.wait(Object.java:502)
   [junit4]   2> 	        at org.apache.solr.core.CloserThread.run(CoreContainer.java:1035)
   [junit4]   2> 	   2) Thread[id=9932, name=TEST-TestSurroundQueryParser.testQueryParser-seed#[ABEA39500764A10D], state=WAITING, group=TGRP-TestSurroundQueryParser]
   [junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
   [junit4]   2> 	        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
   [junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
   [junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870)
   [junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199)
   [junit4]   2> 	        at java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.lock(ReentrantReadWriteLock.java:943)
   [junit4]   2> 	        at org.apache.solr.update.VersionInfo.blockUpdates(VersionInfo.java:119)
   [junit4]   2> 	        at org.apache.solr.update.UpdateLog.onFirstSearcher(UpdateLog.java:1604)
   [junit4]   2> 	        at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1810)
   [junit4]   2> 	        at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1505)
   [junit4]   2> 	        at org.apache.solr.update.DirectUpdateHandler2.commit(DirectUpdateHandler2.java:617)
   [junit4]   2> 	        at org.apache.solr.update.processor.RunUpdateProcessor.processCommit(RunUpdateProcessorFactory.java:95)
   [junit4]   2> 	        at org.apache.solr.update.processor.UpdateRequestProcessor.processCommit(UpdateRequestProcessor.java:64)
   [junit4]   2> 	        at org.apache.solr.update.processor.DistributedUpdateProcessor.doLocalCommit(DistributedUpdateProcessor.java:1635)
   [junit4]   2> 	        at org.apache.solr.update.processor.DistributedUpdateProcessor.processCommit(DistributedUpdateProcessor.java:1612)
   [junit4]   2> 	        at org.apache.solr.update.processor.LogUpdateProcessor.processCommit(LogUpdateProcessorFactory.java:161)
   [junit4]   2> 	        at org.apache.solr.handler.loader.XMLLoader.processUpdate(XMLLoader.java:270)
   [junit4]   2> 	        at org.apache.solr.handler.loader.XMLLoader.load(XMLLoader.java:177)
   [junit4]   2> 	        at org.apache.solr.handler.UpdateRequestHandler$1.load(UpdateRequestHandler.java:98)
   [junit4]   2> 	        at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:74)
   [junit4]   2> 	        at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:143)
   [junit4]   2> 	        at org.apache.solr.core.SolrCore.execute(SolrCore.java:2051)
   [junit4]   2> 	        at org.apache.solr.servlet.DirectSolrConnection.request(DirectSolrConnection.java:131)
   [junit4]   2> 	        at org.apache.solr.util.TestHarness.update(TestHarness.java:267)
   [junit4]   2> 	        at org.apache.solr.util.BaseTestHarness.checkUpdateStatus(BaseTestHarness.java:282)
   [junit4]   2> 	        at org.apache.solr.util.BaseTestHarness.validateUpdate(BaseTestHarness.java:252)
   [junit4]   2> 	        at org.apache.solr.SolrTestCaseJ4.checkUpdateU(SolrTestCaseJ4.java:748)
   [junit4]   2> 	        at org.apache.solr.SolrTestCaseJ4.assertU(SolrTestCaseJ4.java:727)
   [junit4]   2> 	        at org.apache.solr.SolrTestCaseJ4.assertU(SolrTestCaseJ4.java:721)
   [junit4]   2> 	        at org.apache.solr.search.TestSurroundQueryParser.testQueryParser(TestSurroundQueryParser.java:54)
   [junit4]   2> 	        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   2> 	        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   [junit4]   2> 	        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2> 	        at java.lang.reflect.Method.invoke(Method.java:497)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2> 	        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
   [junit4]   2> 	        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2> 	        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
   [junit4]   2> 	        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2> 	        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2> 	        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   2> 	        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2> 	        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2> 	        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   2> 	        at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 	   3) Thread[id=9934, name=searcherExecutor-4207-thread-1, state=WAITING, group=TGRP-TestSurroundQueryParser]
   [junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
   [junit4]   2> 	        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
   [junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
   [junit4]   2> 	        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
   [junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]   2> 	        at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 7242922 T9931 ccr.ThreadLeakControl.tryToInterruptAll Starting to interrupt leaked threads:
   [junit4]   2> 	   1) Thread[id=9935, name=Thread-2836, state=WAITING, group=TGRP-TestSurroundQueryParser]
   [junit4]   2> 	   2) Thread[id=9932, name=TEST-TestSurroundQueryParser.testQueryParser-seed#[ABEA39500764A10D], state=WAITING, group=TGRP-TestSurroundQueryParser]
   [junit4]   2> 	   3) Thread[id=9934, name=searcherExecutor-4207-thread-1, state=WAITING, group=TGRP-TestSurroundQueryParser]
   [junit4]   2> 7245947 T9931 ccr.ThreadLeakControl.tryToInterruptAll SEVERE There are still zombie threads that couldn't be terminated:
   [junit4]   2> 	   1) Thread[id=9935, name=Thread-2836, state=WAITING, group=TGRP-TestSurroundQueryParser]
   [junit4]   2> 	        at java.lang.Object.wait(Native Method)
   [junit4]   2> 	        at java.lang.Object.wait(Object.java:502)
   [junit4]   2> 	        at org.apache.solr.core.CloserThread.run(CoreContainer.java:1035)
   [junit4]   2> 	   2) Thread[id=9932, name=TEST-TestSurroundQueryParser.testQueryParser-seed#[ABEA39500764A10D], state=WAITING, group=TGRP-TestSurroundQueryParser]
   [junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
   [junit4]   2> 	        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
   [junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
   [junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870)
   [junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199)
   [junit4]   2> 	        at java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.lock(ReentrantReadWriteLock.java:943)
   [junit4]   2> 	        at org.apache.solr.update.VersionInfo.blockUpdates(VersionInfo.java:119)
   [junit4]   2> 	        at org.apache.solr.update.UpdateLog.onFirstSearcher(UpdateLog.java:1604)
   [junit4]   2> 	        at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1810)
   [junit4]   2> 	        at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1505)
   [junit4]   2> 	        at org.apache.solr.update.DirectUpdateHandler2.commit(DirectUpdateHandler2.java:617)
   [junit4]   2> 	        at org.apache.solr.update.processor.RunUpdateProcessor.processCommit(RunUpdateProcessorFactory.java:95)
   [junit4]   2> 	        at org.apache.solr.update.processor.UpdateRequestProcessor.processCommit(UpdateRequestProcessor.java:64)
   [junit4]   2> 	        at org.apache.solr.update.processor.DistributedUpdateProcessor.doLocalCommit(DistributedUpdateProcessor.java:1635)
   [junit4]   2> 	        at org.apache.solr.update.processor.DistributedUpdateProcessor.processCommit(DistributedUpdateProcessor.java:1612)
   [junit4]   2> 	        at org.apache.solr.update.processor.LogUpdateProcessor.processCommit(LogUpdateProcessorFactory.java:161)
   [junit4]   2> 	        at org.apache.solr.handler.loader.XMLLoader.processUpdate(XMLLoader.java:270)
   [junit4]   2> 	        at org.apache.solr.handler.loader.XMLLoader.load(XMLLoader.java:177)
   [junit4]   2> 	        at org.apache.solr.handler.UpdateRequestHandler$1.load(UpdateRequestHandler.java:98)
   [junit4]   2> 	        at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:74)
   [junit4]   2> 	        at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:143)
   [junit4]   2> 	        at org.apache.solr.core.SolrCore.execute(SolrCore.java:2051)
   [junit4]   2> 	        at org.apache.solr.servlet.DirectSolrConnection.request(DirectSolrConnection.java:131)
   [junit4]   2> 	        at org.apache.solr.util.TestHarness.update(TestHarness.java:267)
   [junit4]   2> 	        at org.apache.solr.util.BaseTestHarness.checkUpdateStatus(BaseTestHarness.java:282)
   [junit4]   2> 	        at org.apache.solr.util.BaseTestHarness.validateUpdate(BaseTestHarness.java:252)
   [junit4]   2> 	        at org.apache.solr.SolrTestCaseJ4.checkUpdateU(SolrTestCaseJ4.java:748)
   [junit4]   2> 	        at org.apache.solr.SolrTestCaseJ4.assertU(SolrTestCaseJ4.java:727)
   [junit4]   2> 	        at org.apache.solr.SolrTestCaseJ4.assertU(SolrTestCaseJ4.java:721)
   [junit4]   2> 	        at org.apache.solr.search.TestSurroundQueryParser.testQueryParser(TestSurroundQueryParser.java:54)
   [junit4]   2> 	        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   2> 	        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   [junit4]   2> 	        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2> 	        at java.lang.reflect.Method.invoke(Method.java:497)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2> 	        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
   [junit4]   2> 	        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2> 	        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
   [junit4]   2> 	        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2> 	        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2> 	        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   2> 	        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2> 	        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2> 	        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   2> 	        at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 	   3) Thread[id=9934, name=searcherExecutor-4207-thread-1, state=WAITING, group=TGRP-TestSurroundQueryParser]
   [junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
   [junit4]   2> 	        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
   [junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
   [junit4]   2> 	        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
   [junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]   2> 	        at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene50): {_version_=PostingsFormat(name=Memory doPackFST= false), name=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), text_np=PostingsFormat(name=LuceneVarGapDocFreqInterval), id=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), text=PostingsFormat(name=Memory doPackFST= true)}, docValues:{}, sim=RandomSimilarityProvider(queryNorm=false,coord=yes): {}, locale=es_US, timezone=Atlantic/Azores
   [junit4]   2> NOTE: Windows 7 6.1 x86/Oracle Corporation 1.8.0_45 (32-bit)/cpus=3,threads=4,free=316964760,total=536870912
   [junit4]   2> NOTE: All tests run in this JVM: [TestJmxIntegration, TestXIncludeConfig, ZkCLITest, TestBlendedInfixSuggestions, TestBlobHandler, UnloadDistributedZkTest, TestJsonRequest, JSONWriterTest, TestBinaryField, FieldAnalysisRequestHandlerTest, DistributedFacetPivotLongTailTest, SuggesterFSTTest, TestDistributedSearch, AsyncMigrateRouteKeyTest, LeaderInitiatedRecoveryOnCommitTest, TestSchemaNameResource, TestArbitraryIndexDir, DistributedQueryComponentOptimizationTest, UUIDFieldTest, CachingDirectoryFactoryTest, BlockDirectoryTest, TestShortCircuitedRequests, ResponseHeaderTest, CoreAdminCreateDiscoverTest, DistributedSpellCheckComponentTest, CloudMLTQParserTest, DistributedQueryElevationComponentTest, SpatialRPTFieldTypeTest, TestAnalyzeInfixSuggestions, SampleTest, TestCloudManagedSchemaConcurrent, DeleteShardTest, SynonymTokenizerTest, DistanceFunctionTest, RollingRestartTest, HttpPartitionTest, TestLFUCache, OverseerTest, VMParamsZkACLAndCredentialsProvidersTest, TestPerFieldSimilarity, TestIntervalFaceting, ExternalCollectionsTest, TestGroupingSearch, AlternateDirectoryTest, BasicDistributedZkTest, AddSchemaFieldsUpdateProcessorFactoryTest, VersionInfoTest, TestDistributedMissingSort, DistributedFacetPivotLargeTest, TestNamedUpdateProcessors, TestTrackingShardHandlerFactory, UUIDUpdateProcessorFallbackTest, HdfsWriteToMultipleCollectionsTest, TestStressReorder, BadComponentTest, BlockCacheTest, SolrXmlInZkTest, TestComponentsName, ChaosMonkeyNothingIsSafeTest, TestDocumentBuilder, TestQueryTypes, SoftAutoCommitTest, TestNonDefinedSimilarityFactory, TestAnalyzedSuggestions, TestSweetSpotSimilarityFactory, TestConfigSets, AssignTest, TestBulkSchemaConcurrent, TriLevelCompositeIdRoutingTest, DistributedVersionInfoTest, TestClassNameShortening, TestHashPartitioner, TestLRUCache, SuggesterWFSTTest, AnalyticsQueryTest, ZkControllerTest, DefaultValueUpdateProcessorTest, ShardRoutingTest, PeerSyncTest, TestManagedStopFilterFactory, TestImplicitCoreProperties, CustomCollectionTest, TestManagedSchemaFieldTypeResource, PrimUtilsTest, TestFaceting, MultiTermTest, TestSortingResponseWriter, TestCopyFieldCollectionResource, SystemInfoHandlerTest, TestDistribDocBasedVersion, SolrCoreTest, ExitableDirectoryReaderTest, TestJmxMonitoredMap, RecoveryAfterSoftCommitTest, SolrCoreCheckLockOnStartupTest, ClusterStateTest, HdfsLockFactoryTest, IndexSchemaRuntimeFieldTest, TestFastLRUCache, ClusterStateUpdateTest, TestRandomDVFaceting, TestClusterStateMutator, AnalysisErrorHandlingTest, SolrInfoMBeanTest, TestCoreContainer, TestObjectReleaseTracker, RequiredFieldsTest, AddBlockUpdateTest, ScriptEngineTest, TestRandomMergePolicy, CursorMarkTest, SegmentsInfoRequestHandlerTest, URLClassifyProcessorTest, DocValuesMultiTest, SolrIndexSplitterTest, TestSurroundQueryParser]
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestSurroundQueryParser -Dtests.seed=ABEA39500764A10D -Dtests.slow=true -Dtests.locale=es_US -Dtests.timezone=Atlantic/Azores -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
   [junit4] ERROR   0.00s J0 | TestSurroundQueryParser (suite) <<<
   [junit4]    > Throwable #1: java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([ABEA39500764A10D]:0)
   [junit4] Completed [492/494] on J0 in 7245.97s, 1 test, 2 errors <<< FAILURES!

[...truncated 17 lines...]
BUILD FAILED
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\build.xml:526: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\build.xml:474: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\build.xml:61: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\extra-targets.xml:39: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build.xml:229: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\common-build.xml:512: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\common-build.xml:1415: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\common-build.xml:973: There were test failures: 494 suites, 1968 tests, 1 suite-level error, 1 error, 77 ignored (34 assumptions)

Total time: 164 minutes 29 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-trunk-Windows (32bit/jdk1.8.0_45) - Build # 4846 - Still Failing!

Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Windows/4846/
Java: 32bit/jdk1.8.0_45 -server -XX:+UseSerialGC

2 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.highlight.HighlighterTest

Error Message:
Suite timeout exceeded (>= 7200000 msec).

Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
	at __randomizedtesting.SeedInfo.seed([7CC600A9A408E871]:0)


FAILED:  org.apache.solr.highlight.HighlighterTest.payloadFilteringSpanQuery

Error Message:
Test abandoned because suite timeout was reached.

Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
	at __randomizedtesting.SeedInfo.seed([7CC600A9A408E871]:0)




Build Log:
[...truncated 10931 lines...]
   [junit4] Suite: org.apache.solr.highlight.HighlighterTest
   [junit4]   2> Creating dataDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.highlight.HighlighterTest 7CC600A9A408E871-001\init-core-data-001
   [junit4]   2> 1650126 T8069 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (false) and clientAuth (false)
   [junit4]   2> 1650128 T8069 oas.SolrTestCaseJ4.initCore ####initCore
   [junit4]   2> 1650128 T8069 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\'
   [junit4]   2> 1650129 T8069 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/C:/Users/JenkinsSlave/workspace/Lucene-Solr-trunk-Windows/solr/core/src/test-files/solr/collection1/lib/classes/' to classloader
   [junit4]   2> 1650130 T8069 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/C:/Users/JenkinsSlave/workspace/Lucene-Solr-trunk-Windows/solr/core/src/test-files/solr/collection1/lib/README' to classloader
   [junit4]   2> 1650173 T8069 oasc.SolrConfig.refreshRequestParams current version of requestparams : -1
   [junit4]   2> 1650194 T8069 oasc.SolrConfig.<init> Using Lucene MatchVersion: 6.0.0
   [junit4]   2> 1650209 T8069 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1650209 T8069 oass.IndexSchema.readSchema Reading Solr Schema from C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\schema.xml
   [junit4]   2> 1650214 T8069 oass.IndexSchema.readSchema [null] Schema name=test
   [junit4]   2> 1650289 T8069 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 1650297 T8069 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 1650298 T8069 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 1650302 T8069 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 1650304 T8069 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 1650306 T8069 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1650307 T8069 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 1650307 T8069 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 1650307 T8069 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1650308 T8069 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 1650308 T8069 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 1650308 T8069 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1650308 T8069 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr
   [junit4]   2> 1650309 T8069 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\'
   [junit4]   2> 1650322 T8069 oasc.CoreContainer.<init> New CoreContainer 17714728
   [junit4]   2> 1650323 T8069 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\]
   [junit4]   2> 1650323 T8069 oasc.CoreContainer.load loading shared library: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\lib
   [junit4]   2> 1650323 T8069 oasc.SolrResourceLoader.addToClassLoader WARN Can't find (or read) directory to add to classloader: lib (resolved as: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\lib).
   [junit4]   2> 1650330 T8069 oashc.HttpShardHandlerFactory.init created with socketTimeout : 600000,connTimeout : 60000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1650333 T8069 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=30000&connTimeout=30000&retry=true
   [junit4]   2> 1650334 T8069 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1650334 T8069 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1650335 T8069 oasc.CoreContainer.load Node Name: testNode
   [junit4]   2> 1650335 T8069 oasc.CoreContainer.initializeAuthenticationPlugin No authentication plugin used.
   [junit4]   2> 1650344 T8069 oasc.CoreDescriptor.<init> CORE DESCRIPTOR: {name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, loadOnStartup=true, instanceDir=collection1, collection=collection1, absoluteInstDir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\, dataDir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.highlight.HighlighterTest 7CC600A9A408E871-001\init-core-data-001, shard=shard1}
   [junit4]   2> 1650345 T8070 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\'
   [junit4]   2> 1650346 T8070 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/C:/Users/JenkinsSlave/workspace/Lucene-Solr-trunk-Windows/solr/core/src/test-files/solr/collection1/lib/classes/' to classloader
   [junit4]   2> 1650346 T8070 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/C:/Users/JenkinsSlave/workspace/Lucene-Solr-trunk-Windows/solr/core/src/test-files/solr/collection1/lib/README' to classloader
   [junit4]   2> 1650364 T8070 oasc.SolrConfig.refreshRequestParams current version of requestparams : -1
   [junit4]   2> 1650370 T8070 oasc.SolrConfig.<init> Using Lucene MatchVersion: 6.0.0
   [junit4]   2> 1650399 T8070 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1650399 T8070 oass.IndexSchema.readSchema Reading Solr Schema from C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\schema.xml
   [junit4]   2> 1650404 T8070 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 1650474 T8070 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 1650479 T8070 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 1650481 T8070 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 1650485 T8070 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 1650487 T8070 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 1650489 T8070 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1650489 T8070 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 1650489 T8070 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 1650489 T8070 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1650490 T8070 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 1650490 T8070 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 1650490 T8070 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from instancedir C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\
   [junit4]   2> 1650490 T8070 x:collection1 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 1650491 T8070 x:collection1 oasc.SolrCore.<init> [[collection1] ] Opening new SolrCore at [C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\], dataDir=[null]
   [junit4]   2> 1650491 T8070 x:collection1 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1d86cd9
   [junit4]   2> 1650491 T8070 x:collection1 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.highlight.HighlighterTest 7CC600A9A408E871-001\init-core-data-001\
   [junit4]   2> 1650492 T8070 x:collection1 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.highlight.HighlighterTest 7CC600A9A408E871-001\init-core-data-001\index/
   [junit4]   2> 1650492 T8070 x:collection1 oasc.SolrCore.initIndex WARN [collection1] Solr index directory 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.highlight.HighlighterTest 7CC600A9A408E871-001\init-core-data-001\index' doesn't exist. Creating new index...
   [junit4]   2> 1650492 T8070 x:collection1 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.highlight.HighlighterTest 7CC600A9A408E871-001\init-core-data-001\index
   [junit4]   2> 1650492 T8070 x:collection1 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=31, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.15520104308546642]
   [junit4]   2> 1650493 T8070 x:collection1 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@364638 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1b65bf9),segFN=segments_1,generation=1}
   [junit4]   2> 1650493 T8070 x:collection1 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 1650496 T8070 x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 1650496 T8070 x:collection1 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 1650497 T8070 x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe-allfields"
   [junit4]   2> 1650497 T8070 x:collection1 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
   [junit4]   2> 1650497 T8070 x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1650497 T8070 x:collection1 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1650497 T8070 x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "uniq-fields"
   [junit4]   2> 1650497 T8070 x:collection1 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
   [junit4]   2> 1650498 T8070 x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 1650498 T8070 x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1650498 T8070 x:collection1 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1650498 T8070 x:collection1 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 1650502 T8070 x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 1650503 T8070 x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 1650505 T8070 x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 1650505 T8070 x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 1650509 T8070 x:collection1 oasc.PluginBag.createPlugin requestHandler : 'lazy' created with startup=lazy 
   [junit4]   2> 1650510 T8070 x:collection1 oasc.RequestHandlers.initHandlersFromConfig Registered paths: /admin/mbeans,standard,spellCheckCompRH,lazy,/update/json/docs,/admin/luke,spellCheckWithWordbreak_Direct,tvrh,/get,/admin/properties,/update/json,mltrh,/admin/threads,/search-facet-def,/update/csv,/search-facet-invariants,/admin/segments,spellCheckCompRH1,/mlt,/admin/system,/replication,dismax,defaults,/config,/schema,/admin/plugins,/admin/logging,spellCheckWithWordbreak,mock,/admin/ping,/update,/admin/file,/terms,spellCheckCompRH_Direct,/debug/dump
   [junit4]   2> 1650511 T8070 x:collection1 oasc.SolrCore.initDeprecatedSupport WARN solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
   [junit4]   2> 1650511 T8070 x:collection1 oasc.SolrCore.initStatsCache Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 1650513 T8070 x:collection1 oasu.UpdateHandler.<init> Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1650514 T8070 x:collection1 oasu.UpdateLog.init Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1650514 T8070 x:collection1 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 1650515 T8070 x:collection1 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 1650516 T8070 x:collection1 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=25, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
   [junit4]   2> 1650516 T8070 x:collection1 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@364638 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1b65bf9),segFN=segments_1,generation=1}
   [junit4]   2> 1650516 T8070 x:collection1 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 1650517 T8070 x:collection1 oass.SolrIndexSearcher.<init> Opening Searcher@fa69e7[collection1] main
   [junit4]   2> 1650517 T8070 x:collection1 oasu.UpdateLog.onFirstSearcher On first searcher opened, looking up max value of version field
   [junit4]   2> 1650517 T8070 x:collection1 oasu.VersionInfo.getMaxVersionFromIndex Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 1650517 T8070 x:collection1 oasu.VersionInfo.getMaxVersionFromIndex WARN No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 1650517 T8070 x:collection1 oasu.UpdateLog.seedBucketsWithHighestVersion WARN Could not find max version in index or recent updates, using new clock 1502022332634365952
   [junit4]   2> 1650519 T8070 x:collection1 oasu.UpdateLog.seedBucketsWithHighestVersion Took 2 ms to seed version buckets with highest version 1502022332634365952
   [junit4]   2> 1650520 T8070 x:collection1 oasr.RestManager.init Initializing RestManager with initArgs: {storageIO=org.apache.solr.rest.ManagedResourceStorage$InMemoryStorageIO}
   [junit4]   2> 1650520 T8070 x:collection1 oasr.ManagedResourceStorage.load Reading _rest_managed.json using InMemoryStorage
   [junit4]   2> 1650520 T8070 x:collection1 oasr.RestManager.init Initializing 0 registered ManagedResources
   [junit4]   2> 1650520 T8070 x:collection1 oashc.SpellCheckComponent.inform Initializing spell checkers
   [junit4]   2> 1650529 T8070 x:collection1 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
   [junit4]   2> 1650570 T8070 x:collection1 oash.ReplicationHandler.inform Commits will be reserved for  10000
   [junit4]   2> 1650571 T8070 x:collection1 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 1650573 T8069 oas.SolrTestCaseJ4.initCore ####initCore end
   [junit4]   2> 1650575 T8069 oas.SolrTestCaseJ4.setUp ###Starting payloadFilteringSpanQuery
   [junit4]   2> ASYNC  NEW_CORE C3017 name=collection1 org.apache.solr.core.SolrCore@2e282f
   [junit4]   2> 1650577 T8069 C3017 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {deleteByQuery=*:* (-1502022332695183360)} 0 2
   [junit4]   2> 1650583 T8069 C3017 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[0 (1502022332699377664)]} 0 4
   [junit4]   2> 1650584 T8069 C3017 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 1650593 T8069 C3017 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@364638 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1b65bf9),segFN=segments_1,generation=1}
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@364638 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1b65bf9),segFN=segments_2,generation=2}
   [junit4]   2> 1650594 T8069 C3017 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
   [junit4]   2> 1650595 T8069 C3017 oasc.SolrCore.getSearcher WARN [collection1] PERFORMANCE WARNING: Overlapping onDeckSearchers=2
   [junit4]   2> 1650596 T8069 C3017 oass.SolrIndexSearcher.<init> Opening Searcher@1329281[collection1] main
   [junit4]   2> 1650596 T8069 C3017 oasu.UpdateLog.onFirstSearcher On first searcher opened, looking up max value of version field
   [junit4]   2> 1650597 T8071 x:collection1 oashc.SpellCheckComponent$SpellCheckerListener.newSearcher Loading spell index for spellchecker: default
   [junit4]   2> 1650597 T8071 x:collection1 oashc.SpellCheckComponent$SpellCheckerListener.newSearcher Loading spell index for spellchecker: direct
   [junit4]   2> 1650597 T8071 x:collection1 oashc.SpellCheckComponent$SpellCheckerListener.newSearcher Loading spell index for spellchecker: wordbreak
   [junit4]   2> 1650597 T8071 x:collection1 oashc.SpellCheckComponent$SpellCheckerListener.newSearcher Loading spell index for spellchecker: multipleFields
   [junit4]   2> 1650598 T8071 x:collection1 oashc.SpellCheckComponent$SpellCheckerListener.newSearcher Loading spell index for spellchecker: jarowinkler
   [junit4]   2> 1650598 T8071 x:collection1 oashc.SpellCheckComponent$SpellCheckerListener.newSearcher Loading spell index for spellchecker: external
   [junit4]   2> 1650599 T8071 x:collection1 oashc.SpellCheckComponent$SpellCheckerListener.newSearcher Loading spell index for spellchecker: freq
   [junit4]   2> 1650599 T8071 x:collection1 oashc.SpellCheckComponent$SpellCheckerListener.newSearcher Loading spell index for spellchecker: fqcn
   [junit4]   2> 1650599 T8071 x:collection1 oashc.SpellCheckComponent$SpellCheckerListener.newSearcher Loading spell index for spellchecker: perDict
   [junit4]   2> 1650599 T8071 x:collection1 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@fa69e7[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 7257252 T8068 ccr.ThreadLeakControl$2.evaluate WARNING Suite execution timed out: org.apache.solr.highlight.HighlighterTest
   [junit4]   2> 	==== jstack at approximately timeout time ====
   [junit4]   2> 	"Thread-2968" ID=8072 WAITING on java.lang.Object@341078
   [junit4]   2> 		at java.lang.Object.wait(Native Method)
   [junit4]   2> 		- waiting on java.lang.Object@341078
   [junit4]   2> 		at java.lang.Object.wait(Object.java:502)
   [junit4]   2> 		at org.apache.solr.core.CloserThread.run(CoreContainer.java:1035)
   [junit4]   2> 	
   [junit4]   2> 	"searcherExecutor-4057-thread-1" ID=8071 WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@aa4378
   [junit4]   2> 		at sun.misc.Unsafe.park(Native Method)
   [junit4]   2> 		- waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@aa4378
   [junit4]   2> 		at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
   [junit4]   2> 		at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
   [junit4]   2> 		at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]   2> 		at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
   [junit4]   2> 		at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]   2> 		at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]   2> 		at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 	
   [junit4]   2> 	"TEST-HighlighterTest.payloadFilteringSpanQuery-seed#[7CC600A9A408E871]" ID=8069 WAITING on java.util.concurrent.locks.ReentrantReadWriteLock$FairSync@85bc37
   [junit4]   2> 		at sun.misc.Unsafe.park(Native Method)
   [junit4]   2> 		- waiting on java.util.concurrent.locks.ReentrantReadWriteLock$FairSync@85bc37
   [junit4]   2> 		at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
   [junit4]   2> 		at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
   [junit4]   2> 		at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870)
   [junit4]   2> 		at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199)
   [junit4]   2> 		at java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.lock(ReentrantReadWriteLock.java:943)
   [junit4]   2> 		at org.apache.solr.update.VersionInfo.blockUpdates(VersionInfo.java:119)
   [junit4]   2> 		at org.apache.solr.update.UpdateLog.onFirstSearcher(UpdateLog.java:1604)
   [junit4]   2> 		at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1810)
   [junit4]   2> 		at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1505)
   [junit4]   2> 		at org.apache.solr.update.DirectUpdateHandler2.commit(DirectUpdateHandler2.java:617)
   [junit4]   2> 		- locked java.lang.Object@c683c2
   [junit4]   2> 		at org.apache.solr.update.processor.RunUpdateProcessor.processCommit(RunUpdateProcessorFactory.java:95)
   [junit4]   2> 		at org.apache.solr.update.processor.UpdateRequestProcessor.processCommit(UpdateRequestProcessor.java:64)
   [junit4]   2> 		at org.apache.solr.update.processor.DistributedUpdateProcessor.doLocalCommit(DistributedUpdateProcessor.java:1642)
   [junit4]   2> 		at org.apache.solr.update.processor.DistributedUpdateProcessor.processCommit(DistributedUpdateProcessor.java:1619)
   [junit4]   2> 		at org.apache.solr.update.processor.LogUpdateProcessor.processCommit(LogUpdateProcessorFactory.java:161)
   [junit4]   2> 		at org.apache.solr.handler.loader.XMLLoader.processUpdate(XMLLoader.java:270)
   [junit4]   2> 		at org.apache.solr.handler.loader.XMLLoader.load(XMLLoader.java:177)
   [junit4]   2> 		at org.apache.solr.handler.UpdateRequestHandler$1.load(UpdateRequestHandler.java:98)
   [junit4]   2> 		at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:74)
   [junit4]   2> 		at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:143)
   [junit4]   2> 		at org.apache.solr.core.SolrCore.execute(SolrCore.java:2051)
   [junit4]   2> 		at org.apache.solr.servlet.DirectSolrConnection.request(DirectSolrConnection.java:131)
   [junit4]   2> 		at org.apache.solr.util.TestHarness.update(TestHarness.java:267)
   [junit4]   2> 		at org.apache.solr.util.BaseTestHarness.checkUpdateStatus(BaseTestHarness.java:282)
   [junit4]   2> 		at org.apache.solr.util.BaseTestHarness.validateUpdate(BaseTestHarness.java:252)
   [junit4]   2> 		at org.apache.solr.SolrTestCaseJ4.checkUpdateU(SolrTestCaseJ4.java:748)
   [junit4]   2> 		at org.apache.solr.SolrTestCaseJ4.assertU(SolrTestCaseJ4.java:727)
   [junit4]   2> 		at org.apache.solr.SolrTestCaseJ4.assertU(SolrTestCaseJ4.java:721)
   [junit4]   2> 		at org.apache.solr.highlight.HighlighterTest.payloadFilteringSpanQuery(HighlighterTest.java:1105)
   [junit4]   2> 		at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   2> 		at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   [junit4]   2> 		at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2> 		at java.lang.reflect.Method.invoke(Method.java:497)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2> 		at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
   [junit4]   2> 		at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2> 		at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
   [junit4]   2> 		at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2> 		at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2> 		at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 		at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 		at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   2> 		at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2> 		at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2> 		at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   2> 		at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 		Locked synchronizers:
   [junit4]   2> 		- java.util.concurrent.locks.ReentrantLock$FairSync@1600688
   [junit4]   2> 		- java.util.concurrent.locks.ReentrantLock$NonfairSync@e69890
   [junit4]   2> 	
   [junit4]   2> 	"SUITE-HighlighterTest-seed#[7CC600A9A408E871]" ID=8068 RUNNABLE
   [junit4]   2> 		at sun.management.ThreadImpl.dumpThreads0(Native Method)
   [junit4]   2> 		at sun.management.ThreadImpl.dumpAllThreads(ThreadImpl.java:446)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.ThreadLeakControl.formatThreadStacksFull(ThreadLeakControl.java:671)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.ThreadLeakControl.access$900(ThreadLeakControl.java:62)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.ThreadLeakControl$2.evaluate(ThreadLeakControl.java:412)
   [junit4]   2> 		- locked java.lang.Object@23fb76
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:651)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.RandomizedRunner.access$200(RandomizedRunner.java:138)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.RandomizedRunner$1.run(RandomizedRunner.java:568)
   [junit4]   2> 	
   [junit4]   2> 	"TimeLimitedCollector timer thread" ID=5325 TIMED_WAITING
   [junit4]   2> 		at java.lang.Thread.sleep(Native Method)
   [junit4]   2> 		at org.apache.lucene.search.TimeLimitingCollector$TimerThread.run(TimeLimitingCollector.java:256)
   [junit4]   2> 	
   [junit4]   2> 	"JUnit4-serializer-daemon" ID=10 TIMED_WAITING
   [junit4]   2> 		at java.lang.Thread.sleep(Native Method)
   [junit4]   2> 		at com.carrotsearch.ant.tasks.junit4.events.Serializer$1.run(Serializer.java:47)
   [junit4]   2> 	
   [junit4]   2> 	"Attach Listener" ID=5 RUNNABLE
   [junit4]   2> 	
   [junit4]   2> 	"Signal Dispatcher" ID=4 RUNNABLE
   [junit4]   2> 	
   [junit4]   2> 	"Finalizer" ID=3 WAITING on java.lang.ref.ReferenceQueue$Lock@3a7078
   [junit4]   2> 		at java.lang.Object.wait(Native Method)
   [junit4]   2> 		- waiting on java.lang.ref.ReferenceQueue$Lock@3a7078
   [junit4]   2> 		at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:143)
   [junit4]   2> 		at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:164)
   [junit4]   2> 		at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:209)
   [junit4]   2> 	
   [junit4]   2> 	"Reference Handler" ID=2 WAITING on java.lang.ref.Reference$Lock@60a32b
   [junit4]   2> 		at java.lang.Object.wait(Native Method)
   [junit4]   2> 		- waiting on java.lang.ref.Reference$Lock@60a32b
   [junit4]   2> 		at java.lang.Object.wait(Object.java:502)
   [junit4]   2> 		at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:157)
   [junit4]   2> 	
   [junit4]   2> 	"main" ID=1 WAITING on com.carrotsearch.randomizedtesting.RandomizedRunner$1@1974943
   [junit4]   2> 		at java.lang.Object.wait(Native Method)
   [junit4]   2> 		- waiting on com.carrotsearch.randomizedtesting.RandomizedRunner$1@1974943
   [junit4]   2> 		at java.lang.Thread.join(Thread.java:1245)
   [junit4]   2> 		at java.lang.Thread.join(Thread.java:1319)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:578)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.RandomizedRunner.run(RandomizedRunner.java:444)
   [junit4]   2> 		at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.execute(SlaveMain.java:199)
   [junit4]   2> 		at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.main(SlaveMain.java:310)
   [junit4]   2> 		at com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe.main(SlaveMainSafe.java:12)
   [junit4]   2> 	
   [junit4]   2> 	^^==============================================
   [junit4]   2> 	
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=HighlighterTest -Dtests.method=payloadFilteringSpanQuery -Dtests.seed=7CC600A9A408E871 -Dtests.slow=true -Dtests.locale=es_NI -Dtests.timezone=America/Moncton -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR   7257s J0 | HighlighterTest.payloadFilteringSpanQuery <<<
   [junit4]    > Throwable #1: java.lang.Exception: Test abandoned because suite timeout was reached.
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([7CC600A9A408E871]:0)
   [junit4]   2> 7257277 T8068 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 3 leaked thread(s).
   [junit4]   2> 7277320 T8068 ccr.ThreadLeakControl.checkThreadLeaks SEVERE 3 threads leaked from SUITE scope at org.apache.solr.highlight.HighlighterTest: 
   [junit4]   2> 	   1) Thread[id=8069, name=TEST-HighlighterTest.payloadFilteringSpanQuery-seed#[7CC600A9A408E871], state=WAITING, group=TGRP-HighlighterTest]
   [junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
   [junit4]   2> 	        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
   [junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
   [junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870)
   [junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199)
   [junit4]   2> 	        at java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.lock(ReentrantReadWriteLock.java:943)
   [junit4]   2> 	        at org.apache.solr.update.VersionInfo.blockUpdates(VersionInfo.java:119)
   [junit4]   2> 	        at org.apache.solr.update.UpdateLog.onFirstSearcher(UpdateLog.java:1604)
   [junit4]   2> 	        at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1810)
   [junit4]   2> 	        at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1505)
   [junit4]   2> 	        at org.apache.solr.update.DirectUpdateHandler2.commit(DirectUpdateHandler2.java:617)
   [junit4]   2> 	        at org.apache.solr.update.processor.RunUpdateProcessor.processCommit(RunUpdateProcessorFactory.java:95)
   [junit4]   2> 	        at org.apache.solr.update.processor.UpdateRequestProcessor.processCommit(UpdateRequestProcessor.java:64)
   [junit4]   2> 	        at org.apache.solr.update.processor.DistributedUpdateProcessor.doLocalCommit(DistributedUpdateProcessor.java:1642)
   [junit4]   2> 	        at org.apache.solr.update.processor.DistributedUpdateProcessor.processCommit(DistributedUpdateProcessor.java:1619)
   [junit4]   2> 	        at org.apache.solr.update.processor.LogUpdateProcessor.processCommit(LogUpdateProcessorFactory.java:161)
   [junit4]   2> 	        at org.apache.solr.handler.loader.XMLLoader.processUpdate(XMLLoader.java:270)
   [junit4]   2> 	        at org.apache.solr.handler.loader.XMLLoader.load(XMLLoader.java:177)
   [junit4]   2> 	        at org.apache.solr.handler.UpdateRequestHandler$1.load(UpdateRequestHandler.java:98)
   [junit4]   2> 	        at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:74)
   [junit4]   2> 	        at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:143)
   [junit4]   2> 	        at org.apache.solr.core.SolrCore.execute(SolrCore.java:2051)
   [junit4]   2> 	        at org.apache.solr.servlet.DirectSolrConnection.request(DirectSolrConnection.java:131)
   [junit4]   2> 	        at org.apache.solr.util.TestHarness.update(TestHarness.java:267)
   [junit4]   2> 	        at org.apache.solr.util.BaseTestHarness.checkUpdateStatus(BaseTestHarness.java:282)
   [junit4]   2> 	        at org.apache.solr.util.BaseTestHarness.validateUpdate(BaseTestHarness.java:252)
   [junit4]   2> 	        at org.apache.solr.SolrTestCaseJ4.checkUpdateU(SolrTestCaseJ4.java:748)
   [junit4]   2> 	        at org.apache.solr.SolrTestCaseJ4.assertU(SolrTestCaseJ4.java:727)
   [junit4]   2> 	        at org.apache.solr.SolrTestCaseJ4.assertU(SolrTestCaseJ4.java:721)
   [junit4]   2> 	        at org.apache.solr.highlight.HighlighterTest.payloadFilteringSpanQuery(HighlighterTest.java:1105)
   [junit4]   2> 	        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   2> 	        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   [junit4]   2> 	        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2> 	        at java.lang.reflect.Method.invoke(Method.java:497)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2> 	        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
   [junit4]   2> 	        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2> 	        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
   [junit4]   2> 	        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2> 	        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2> 	        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   2> 	        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2> 	        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2> 	        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   2> 	        at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 	   2) Thread[id=8071, name=searcherExecutor-4057-thread-1, state=WAITING, group=TGRP-HighlighterTest]
   [junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
   [junit4]   2> 	        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
   [junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
   [junit4]   2> 	        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
   [junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]   2> 	        at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 	   3) Thread[id=8072, name=Thread-2968, state=WAITING, group=TGRP-HighlighterTest]
   [junit4]   2> 	        at java.lang.Object.wait(Native Method)
   [junit4]   2> 	        at java.lang.Object.wait(Object.java:502)
   [junit4]   2> 	        at org.apache.solr.core.CloserThread.run(CoreContainer.java:1035)
   [junit4]   2> 7277322 T8068 ccr.ThreadLeakControl.tryToInterruptAll Starting to interrupt leaked threads:
   [junit4]   2> 	   1) Thread[id=8069, name=TEST-HighlighterTest.payloadFilteringSpanQuery-seed#[7CC600A9A408E871], state=WAITING, group=TGRP-HighlighterTest]
   [junit4]   2> 	   2) Thread[id=8071, name=searcherExecutor-4057-thread-1, state=WAITING, group=TGRP-HighlighterTest]
   [junit4]   2> 	   3) Thread[id=8072, name=Thread-2968, state=WAITING, group=TGRP-HighlighterTest]
   [junit4]   2> 7280855 T8068 ccr.ThreadLeakControl.tryToInterruptAll SEVERE There are still zombie threads that couldn't be terminated:
   [junit4]   2> 	   1) Thread[id=8069, name=TEST-HighlighterTest.payloadFilteringSpanQuery-seed#[7CC600A9A408E871], state=WAITING, group=TGRP-HighlighterTest]
   [junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
   [junit4]   2> 	        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
   [junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
   [junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870)
   [junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199)
   [junit4]   2> 	        at java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.lock(ReentrantReadWriteLock.java:943)
   [junit4]   2> 	        at org.apache.solr.update.VersionInfo.blockUpdates(VersionInfo.java:119)
   [junit4]   2> 	        at org.apache.solr.update.UpdateLog.onFirstSearcher(UpdateLog.java:1604)
   [junit4]   2> 	        at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1810)
   [junit4]   2> 	        at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1505)
   [junit4]   2> 	        at org.apache.solr.update.DirectUpdateHandler2.commit(DirectUpdateHandler2.java:617)
   [junit4]   2> 	        at org.apache.solr.update.processor.RunUpdateProcessor.processCommit(RunUpdateProcessorFactory.java:95)
   [junit4]   2> 	        at org.apache.solr.update.processor.UpdateRequestProcessor.processCommit(UpdateRequestProcessor.java:64)
   [junit4]   2> 	        at org.apache.solr.update.processor.DistributedUpdateProcessor.doLocalCommit(DistributedUpdateProcessor.java:1642)
   [junit4]   2> 	        at org.apache.solr.update.processor.DistributedUpdateProcessor.processCommit(DistributedUpdateProcessor.java:1619)
   [junit4]   2> 	        at org.apache.solr.update.processor.LogUpdateProcessor.processCommit(LogUpdateProcessorFactory.java:161)
   [junit4]   2> 	        at org.apache.solr.handler.loader.XMLLoader.processUpdate(XMLLoader.java:270)
   [junit4]   2> 	        at org.apache.solr.handler.loader.XMLLoader.load(XMLLoader.java:177)
   [junit4]   2> 	        at org.apache.solr.handler.UpdateRequestHandler$1.load(UpdateRequestHandler.java:98)
   [junit4]   2> 	        at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:74)
   [junit4]   2> 	        at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:143)
   [junit4]   2> 	        at org.apache.solr.core.SolrCore.execute(SolrCore.java:2051)
   [junit4]   2> 	        at org.apache.solr.servlet.DirectSolrConnection.request(DirectSolrConnection.java:131)
   [junit4]   2> 	        at org.apache.solr.util.TestHarness.update(TestHarness.java:267)
   [junit4]   2> 	        at org.apache.solr.util.BaseTestHarness.checkUpdateStatus(BaseTestHarness.java:282)
   [junit4]   2> 	        at org.apache.solr.util.BaseTestHarness.validateUpdate(BaseTestHarness.java:252)
   [junit4]   2> 	        at org.apache.solr.SolrTestCaseJ4.checkUpdateU(SolrTestCaseJ4.java:748)
   [junit4]   2> 	        at org.apache.solr.SolrTestCaseJ4.assertU(SolrTestCaseJ4.java:727)
   [junit4]   2> 	        at org.apache.solr.SolrTestCaseJ4.assertU(SolrTestCaseJ4.java:721)
   [junit4]   2> 	        at org.apache.solr.highlight.HighlighterTest.payloadFilteringSpanQuery(HighlighterTest.java:1105)
   [junit4]   2> 	        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   2> 	        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   [junit4]   2> 	        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2> 	        at java.lang.reflect.Method.invoke(Method.java:497)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2> 	        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
   [junit4]   2> 	        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2> 	        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
   [junit4]   2> 	        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2> 	        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2> 	        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   2> 	        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2> 	        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2> 	        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   2> 	        at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 	   2) Thread[id=8071, name=searcherExecutor-4057-thread-1, state=WAITING, group=TGRP-HighlighterTest]
   [junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
   [junit4]   2> 	        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
   [junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
   [junit4]   2> 	        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
   [junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]   2> 	        at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 	   3) Thread[id=8072, name=Thread-2968, state=WAITING, group=TGRP-HighlighterTest]
   [junit4]   2> 	        at java.lang.Object.wait(Native Method)
   [junit4]   2> 	        at java.lang.Object.wait(Object.java:502)
   [junit4]   2> 	        at org.apache.solr.core.CloserThread.run(CoreContainer.java:1035)
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene50): {range_facet_l_dv=PostingsFormat(name=Memory doPackFST= true), _version_=FSTOrd50, multiDefault=PostingsFormat(name=LuceneVarGapFixedInterval), intDefault=FSTOrd50, payloadDelimited=BlockTreeOrds(blocksize=128), id=PostingsFormat(name=Memory doPackFST= true), range_facet_i_dv=FSTOrd50, range_facet_l=FSTOrd50, timestamp=FSTOrd50}, docValues:{range_facet_l_dv=DocValuesFormat(name=Lucene50), range_facet_i_dv=DocValuesFormat(name=Direct), timestamp=DocValuesFormat(name=Direct)}, sim=RandomSimilarityProvider(queryNorm=true,coord=crazy): {}, locale=es_NI, timezone=America/Moncton
   [junit4]   2> NOTE: Windows 7 6.1 x86/Oracle Corporation 1.8.0_45 (32-bit)/cpus=3,threads=4,free=166405240,total=377364480
   [junit4]   2> NOTE: All tests run in this JVM: [CoreMergeIndexesAdminHandlerTest, DistanceFunctionTest, NoCacheHeaderTest, TestConfigOverlay, TestCloudManagedSchema, TestSchemaManager, SimpleMLTQParserTest, TestFieldResource, TestTrackingShardHandlerFactory, TestDefaultStatsCache, TestCSVLoader, PathHierarchyTokenizerFactoryTest, ConnectionManagerTest, TestUpdate, TestComplexPhraseQParserPlugin, DefaultValueUpdateProcessorTest, ClusterStateTest, TestAnalyzedSuggestions, UUIDFieldTest, MultiThreadedOCPTest, TestSchemaNameResource, ShowFileRequestHandlerTest, TestComponentsName, ResponseLogComponentTest, TestFreeTextSuggestions, FacetPivotSmallTest, TestLRUCache, TestSort, InfoHandlerTest, DirectUpdateHandlerTest, TestPartialUpdateDeduplication, TestIndexingPerformance, BaseCdcrDistributedZkTest, OverriddenZkACLAndCredentialsProvidersTest, TestFaceting, CacheHeaderTest, BadComponentTest, TestOverriddenPrefixQueryForCustomFieldType, DistributedSpellCheckComponentTest, TestAuthenticationFramework, DeleteReplicaTest, SolrInfoMBeanTest, TestFastLRUCache, TestRawResponseWriter, TestBM25SimilarityFactory, ClusterStateUpdateTest, ChangedSchemaMergeTest, TestDistributedMissingSort, MergeStrategyTest, DistanceUnitsTest, StressHdfsTest, BasicDistributedZkTest, DocumentAnalysisRequestHandlerTest, TestHighFrequencyDictionaryFactory, ZkStateWriterTest, TestDocBasedVersionConstraints, TestCloudSchemaless, SampleTest, TestStressVersions, TestStressLucene, TestCSVResponseWriter, TestRemoveLastDynamicCopyField, HdfsSyncSliceTest, CollectionReloadTest, TestReRankQParserPlugin, TestFileDictionaryLookup, NumericFieldsTest, JSONWriterTest, DOMUtilTest, TestCharFilters, TestSolrConfigHandlerConcurrent, TestDynamicFieldResource, TestSurroundQueryParser, CoreAdminHandlerTest, PingRequestHandlerTest, TestInitQParser, OverseerTest, RulesTest, UniqFieldsUpdateProcessorFactoryTest, TestArbitraryIndexDir, TestPhraseSuggestions, RequestHandlersTest, DistributedFacetPivotWhiteBoxTest, ReplicationFactorTest, TestQueryTypes, TestJsonFacets, DistribCursorPagingTest, AlternateDirectoryTest, TestReloadAndDeleteDocs, LeaderFailoverAfterPartitionTest, TestElisionMultitermQuery, TestGroupingSearch, TestRandomDVFaceting, TestChildDocTransformer, SimpleCollectionCreateDeleteTest, TestExactSharedStatsCache, AnalysisAfterCoreReloadTest, SoftAutoCommitTest, PrimUtilsTest, TestFieldSortValues, DocValuesTest, IndexSchemaRuntimeFieldTest, QueryResultKeyTest, TestLeaderElectionZkExpiry, TestStressRecovery, TestOrdValues, TestSchemaVersionResource, TestConfigSets, TestManagedSchema, RuleEngineTest, DistributedQueryElevationComponentTest, TestCustomSort, DocExpirationUpdateProcessorFactoryTest, DirectSolrConnectionTest, DistributedQueryComponentCustomSortTest, ZkControllerTest, TestHashQParserPlugin, TestFieldTypeCollectionResource, TestQueryUtils, ConcurrentDeleteAndCreateCollectionTest, SuggesterWFSTTest, TestExtendedDismaxParser, UnloadDistributedZkTest, SyncSliceTest, LeaderElectionIntegrationTest, RecoveryZkTest, TestReplicationHandler, ZkSolrClientTest, ShardRoutingCustomTest, TestDistributedSearch, HighlighterTest]
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=HighlighterTest -Dtests.seed=7CC600A9A408E871 -Dtests.slow=true -Dtests.locale=es_NI -Dtests.timezone=America/Moncton -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR   0.00s J0 | HighlighterTest (suite) <<<
   [junit4]    > Throwable #1: java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([7CC600A9A408E871]:0)
   [junit4] Completed [502/502] on J0 in 7280.90s, 1 test, 2 errors <<< FAILURES!

[...truncated 11 lines...]
BUILD FAILED
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\build.xml:526: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\build.xml:474: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\build.xml:61: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\extra-targets.xml:39: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build.xml:229: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\common-build.xml:512: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\common-build.xml:1415: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\common-build.xml:973: There were test failures: 502 suites, 1951 tests, 1 suite-level error, 1 error, 66 ignored (34 assumptions)

Total time: 166 minutes 43 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-trunk-Windows (64bit/jdk1.8.0_45) - Build # 4845 - Still Failing!

Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Windows/4845/
Java: 64bit/jdk1.8.0_45 -XX:+UseCompressedOops -XX:+UseSerialGC

2 tests failed.
FAILED:  org.apache.solr.ConvertedLegacyTest.testABunchOfConvertedStuff

Error Message:
Test abandoned because suite timeout was reached.

Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
	at __randomizedtesting.SeedInfo.seed([9A942A4156E3F0E8]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.ConvertedLegacyTest

Error Message:
Suite timeout exceeded (>= 7200000 msec).

Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
	at __randomizedtesting.SeedInfo.seed([9A942A4156E3F0E8]:0)




Build Log:
[...truncated 10735 lines...]
   [junit4] Suite: org.apache.solr.ConvertedLegacyTest
   [junit4]   2> Creating dataDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J1\temp\solr.ConvertedLegacyTest 9A942A4156E3F0E8-001\init-core-data-001
   [junit4]   2> 793096 T5225 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (false) and clientAuth (false)
   [junit4]   2> 793099 T5225 oas.SolrTestCaseJ4.initCore ####initCore
   [junit4]   2> 793099 T5225 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\'
   [junit4]   2> 793100 T5225 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/C:/Users/JenkinsSlave/workspace/Lucene-Solr-trunk-Windows/solr/core/src/test-files/solr/collection1/lib/classes/' to classloader
   [junit4]   2> 793101 T5225 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/C:/Users/JenkinsSlave/workspace/Lucene-Solr-trunk-Windows/solr/core/src/test-files/solr/collection1/lib/README' to classloader
   [junit4]   2> 793165 T5225 oasc.SolrConfig.refreshRequestParams current version of requestparams : -1
   [junit4]   2> 793177 T5225 oasc.SolrConfig.<init> Using Lucene MatchVersion: 6.0.0
   [junit4]   2> 793207 T5225 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 793208 T5225 oass.IndexSchema.readSchema Reading Solr Schema from C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\schema.xml
   [junit4]   2> 793214 T5225 oass.IndexSchema.readSchema [null] Schema name=test
   [junit4]   2> 793336 T5225 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 793343 T5225 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 793344 T5225 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 793349 T5225 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 793351 T5225 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 793354 T5225 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 793355 T5225 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 793356 T5225 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 793356 T5225 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 793356 T5225 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 793356 T5225 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 793357 T5225 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 793357 T5225 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr
   [junit4]   2> 793357 T5225 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\'
   [junit4]   2> 793374 T5225 oasc.CoreContainer.<init> New CoreContainer 503254736
   [junit4]   2> 793374 T5225 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\]
   [junit4]   2> 793374 T5225 oasc.CoreContainer.load loading shared library: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\lib
   [junit4]   2> 793374 T5225 oasc.SolrResourceLoader.addToClassLoader WARN Can't find (or read) directory to add to classloader: lib (resolved as: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\lib).
   [junit4]   2> 793382 T5225 oashc.HttpShardHandlerFactory.init created with socketTimeout : 600000,connTimeout : 60000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 793386 T5225 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=30000&connTimeout=30000&retry=true
   [junit4]   2> 793387 T5225 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 793387 T5225 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 793387 T5225 oasc.CoreContainer.load Node Name: testNode
   [junit4]   2> 793387 T5225 oasc.CoreContainer.initializeAuthenticationPlugin No authentication plugin used.
   [junit4]   2> 793389 T5225 oasc.CoreDescriptor.<init> CORE DESCRIPTOR: {name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, loadOnStartup=true, instanceDir=collection1, collection=collection1, absoluteInstDir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\, dataDir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J1\temp\solr.ConvertedLegacyTest 9A942A4156E3F0E8-001\init-core-data-001, shard=shard1}
   [junit4]   2> 793391 T5226 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\'
   [junit4]   2> 793393 T5226 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/C:/Users/JenkinsSlave/workspace/Lucene-Solr-trunk-Windows/solr/core/src/test-files/solr/collection1/lib/classes/' to classloader
   [junit4]   2> 793393 T5226 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/C:/Users/JenkinsSlave/workspace/Lucene-Solr-trunk-Windows/solr/core/src/test-files/solr/collection1/lib/README' to classloader
   [junit4]   2> 793424 T5226 oasc.SolrConfig.refreshRequestParams current version of requestparams : -1
   [junit4]   2> 793438 T5226 oasc.SolrConfig.<init> Using Lucene MatchVersion: 6.0.0
   [junit4]   2> 793470 T5226 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 793470 T5226 oass.IndexSchema.readSchema Reading Solr Schema from C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\schema.xml
   [junit4]   2> 793476 T5226 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 793622 T5226 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 793630 T5226 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 793631 T5226 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 793639 T5226 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 793642 T5226 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 793645 T5226 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 793646 T5226 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 793646 T5226 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 793646 T5226 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 793647 T5226 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 793647 T5226 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 793647 T5226 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from instancedir C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\
   [junit4]   2> 793647 T5226 x:collection1 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 793648 T5226 x:collection1 oasc.SolrCore.<init> [[collection1] ] Opening new SolrCore at [C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\], dataDir=[null]
   [junit4]   2> 793648 T5226 x:collection1 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@45d493d3
   [junit4]   2> 793648 T5226 x:collection1 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J1\temp\solr.ConvertedLegacyTest 9A942A4156E3F0E8-001\init-core-data-001\
   [junit4]   2> 793649 T5226 x:collection1 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J1\temp\solr.ConvertedLegacyTest 9A942A4156E3F0E8-001\init-core-data-001\index/
   [junit4]   2> 793649 T5226 x:collection1 oasc.SolrCore.initIndex WARN [collection1] Solr index directory 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J1\temp\solr.ConvertedLegacyTest 9A942A4156E3F0E8-001\init-core-data-001\index' doesn't exist. Creating new index...
   [junit4]   2> 793649 T5226 x:collection1 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J1\temp\solr.ConvertedLegacyTest 9A942A4156E3F0E8-001\init-core-data-001\index
   [junit4]   2> 793650 T5226 x:collection1 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=43, maxMergeAtOnceExplicit=34, maxMergedSegmentMB=8.27734375, floorSegmentMB=1.283203125, forceMergeDeletesPctAllowed=1.5521181997010025, segmentsPerTier=41.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 793652 T5226 x:collection1 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@4553dd1a lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@15b714ef),segFN=segments_1,generation=1}
   [junit4]   2> 793652 T5226 x:collection1 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 793658 T5226 x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 793658 T5226 x:collection1 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 793658 T5226 x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe-allfields"
   [junit4]   2> 793659 T5226 x:collection1 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
   [junit4]   2> 793659 T5226 x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 793659 T5226 x:collection1 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 793659 T5226 x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "uniq-fields"
   [junit4]   2> 793660 T5226 x:collection1 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
   [junit4]   2> 793660 T5226 x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 793660 T5226 x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 793661 T5226 x:collection1 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 793661 T5226 x:collection1 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 793669 T5226 x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 793671 T5226 x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 793673 T5226 x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 793675 T5226 x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 793681 T5226 x:collection1 oasc.PluginBag.createPlugin requestHandler : 'lazy' created with startup=lazy 
   [junit4]   2> 793684 T5226 x:collection1 oasc.RequestHandlers.initHandlersFromConfig Registered paths: /admin/mbeans,standard,spellCheckCompRH,lazy,/update/json/docs,/admin/luke,spellCheckWithWordbreak_Direct,tvrh,/get,/admin/properties,/update/json,mltrh,/admin/threads,/search-facet-def,/update/csv,/search-facet-invariants,/admin/segments,spellCheckCompRH1,/mlt,/admin/system,/replication,dismax,defaults,/config,/schema,/admin/plugins,/admin/logging,spellCheckWithWordbreak,mock,/admin/ping,/update,/admin/file,/terms,spellCheckCompRH_Direct,/debug/dump
   [junit4]   2> 793685 T5226 x:collection1 oasc.SolrCore.initDeprecatedSupport WARN solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
   [junit4]   2> 793685 T5226 x:collection1 oasc.SolrCore.initStatsCache Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 793686 T5226 x:collection1 oasu.UpdateHandler.<init> Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 793687 T5226 x:collection1 oasu.UpdateLog.init Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 793688 T5226 x:collection1 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 793688 T5226 x:collection1 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 793690 T5226 x:collection1 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=47, maxMergeAtOnceExplicit=14, maxMergedSegmentMB=84.15234375, floorSegmentMB=1.90234375, forceMergeDeletesPctAllowed=8.456899662584219, segmentsPerTier=21.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 793690 T5226 x:collection1 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@4553dd1a lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@15b714ef),segFN=segments_1,generation=1}
   [junit4]   2> 793690 T5226 x:collection1 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 793690 T5226 x:collection1 oass.SolrIndexSearcher.<init> Opening Searcher@74b0f690[collection1] main
   [junit4]   2> 793691 T5226 x:collection1 oasu.UpdateLog.onFirstSearcher On first searcher opened, looking up max value of version field
   [junit4]   2> 793691 T5226 x:collection1 oasu.VersionInfo.getMaxVersionFromIndex Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 793691 T5226 x:collection1 oasu.VersionInfo.getMaxVersionFromIndex WARN No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 793691 T5226 x:collection1 oasu.UpdateLog.seedBucketsWithHighestVersion WARN Could not find max version in index or recent updates, using new clock 1501993776339484672
   [junit4]   2> 793693 T5226 x:collection1 oasu.UpdateLog.seedBucketsWithHighestVersion Took 2 ms to seed version buckets with highest version 1501993776339484672
   [junit4]   2> 793693 T5226 x:collection1 oasr.RestManager.init Initializing RestManager with initArgs: {storageIO=org.apache.solr.rest.ManagedResourceStorage$InMemoryStorageIO}
   [junit4]   2> 793694 T5226 x:collection1 oasr.ManagedResourceStorage.load Reading _rest_managed.json using InMemoryStorage
   [junit4]   2> 793694 T5226 x:collection1 oasr.RestManager.init Initializing 0 registered ManagedResources
   [junit4]   2> 793694 T5226 x:collection1 oashc.SpellCheckComponent.inform Initializing spell checkers
   [junit4]   2> 793705 T5226 x:collection1 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
   [junit4]   2> 793762 T5226 x:collection1 oash.ReplicationHandler.inform Commits will be reserved for  10000
   [junit4]   2> 793762 T5227 x:collection1 oashc.SpellCheckComponent$SpellCheckerListener.newSearcher Loading spell index for spellchecker: default
   [junit4]   2> 793762 T5226 x:collection1 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 793762 T5227 x:collection1 oashc.SpellCheckComponent$SpellCheckerListener.newSearcher Loading spell index for spellchecker: direct
   [junit4]   2> 793762 T5227 x:collection1 oashc.SpellCheckComponent$SpellCheckerListener.newSearcher Loading spell index for spellchecker: wordbreak
   [junit4]   2> 793763 T5227 x:collection1 oashc.SpellCheckComponent$SpellCheckerListener.newSearcher Loading spell index for spellchecker: multipleFields
   [junit4]   2> 793764 T5225 oas.SolrTestCaseJ4.initCore ####initCore end
   [junit4]   2> 793764 T5227 x:collection1 oashc.SpellCheckComponent$SpellCheckerListener.newSearcher Loading spell index for spellchecker: jarowinkler
   [junit4]   2> 793767 T5225 oas.SolrTestCaseJ4.setUp ###Starting testABunchOfConvertedStuff
   [junit4]   2> ASYNC  NEW_CORE C1240 name=collection1 org.apache.solr.core.SolrCore@7b99db06
   [junit4]   2> 793768 T5225 C1240 oasu.DirectUpdateHandler2.commit start commit{,optimize=true,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 793768 T5225 C1240 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
   [junit4]   2> 793768 T5225 C1240 oasc.SolrCore.getSearcher WARN [collection1] PERFORMANCE WARNING: Overlapping onDeckSearchers=2
   [junit4]   2> 793769 T5225 C1240 oasc.SolrCore.openNewSearcher SolrIndexSearcher has not changed - not re-opening: org.apache.solr.search.SolrIndexSearcher
   [junit4]   2> 793769 T5225 C1240 oasu.UpdateLog.onFirstSearcher On first searcher opened, looking up max value of version field
   [junit4]   2> 793768 T5227 x:collection1 oashc.SpellCheckComponent$SpellCheckerListener.newSearcher Loading spell index for spellchecker: external
   [junit4]   2> 793770 T5227 x:collection1 oashc.SpellCheckComponent$SpellCheckerListener.newSearcher Loading spell index for spellchecker: freq
   [junit4]   2> 793770 T5227 x:collection1 oashc.SpellCheckComponent$SpellCheckerListener.newSearcher Loading spell index for spellchecker: fqcn
   [junit4]   2> 793770 T5227 x:collection1 oashc.SpellCheckComponent$SpellCheckerListener.newSearcher Loading spell index for spellchecker: perDict
   [junit4]   2> 793772 T5227 x:collection1 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@74b0f690[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 7246119 T5224 ccr.ThreadLeakControl$2.evaluate WARNING Suite execution timed out: org.apache.solr.ConvertedLegacyTest
   [junit4]   2> 	==== jstack at approximately timeout time ====
   [junit4]   2> 	"Thread-1383" ID=5228 WAITING on java.lang.Object@452b1c89
   [junit4]   2> 		at java.lang.Object.wait(Native Method)
   [junit4]   2> 		- waiting on java.lang.Object@452b1c89
   [junit4]   2> 		at java.lang.Object.wait(Object.java:502)
   [junit4]   2> 		at org.apache.solr.core.CloserThread.run(CoreContainer.java:1035)
   [junit4]   2> 	
   [junit4]   2> 	"searcherExecutor-2318-thread-1" ID=5227 WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@4033308d
   [junit4]   2> 		at sun.misc.Unsafe.park(Native Method)
   [junit4]   2> 		- waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@4033308d
   [junit4]   2> 		at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
   [junit4]   2> 		at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
   [junit4]   2> 		at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]   2> 		at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
   [junit4]   2> 		at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]   2> 		at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]   2> 		at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 	
   [junit4]   2> 	"TEST-ConvertedLegacyTest.testABunchOfConvertedStuff-seed#[9A942A4156E3F0E8]" ID=5225 WAITING on java.util.concurrent.locks.ReentrantReadWriteLock$FairSync@7c6a18b3
   [junit4]   2> 		at sun.misc.Unsafe.park(Native Method)
   [junit4]   2> 		- waiting on java.util.concurrent.locks.ReentrantReadWriteLock$FairSync@7c6a18b3
   [junit4]   2> 		at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
   [junit4]   2> 		at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
   [junit4]   2> 		at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870)
   [junit4]   2> 		at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199)
   [junit4]   2> 		at java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.lock(ReentrantReadWriteLock.java:943)
   [junit4]   2> 		at org.apache.solr.update.VersionInfo.blockUpdates(VersionInfo.java:119)
   [junit4]   2> 		at org.apache.solr.update.UpdateLog.onFirstSearcher(UpdateLog.java:1604)
   [junit4]   2> 		at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1810)
   [junit4]   2> 		at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1505)
   [junit4]   2> 		at org.apache.solr.update.DirectUpdateHandler2.commit(DirectUpdateHandler2.java:617)
   [junit4]   2> 		- locked java.lang.Object@74241dbf
   [junit4]   2> 		at org.apache.solr.update.processor.RunUpdateProcessor.processCommit(RunUpdateProcessorFactory.java:95)
   [junit4]   2> 		at org.apache.solr.update.processor.UpdateRequestProcessor.processCommit(UpdateRequestProcessor.java:64)
   [junit4]   2> 		at org.apache.solr.update.processor.DistributedUpdateProcessor.doLocalCommit(DistributedUpdateProcessor.java:1642)
   [junit4]   2> 		at org.apache.solr.update.processor.DistributedUpdateProcessor.processCommit(DistributedUpdateProcessor.java:1619)
   [junit4]   2> 		at org.apache.solr.update.processor.LogUpdateProcessor.processCommit(LogUpdateProcessorFactory.java:161)
   [junit4]   2> 		at org.apache.solr.handler.loader.XMLLoader.processUpdate(XMLLoader.java:270)
   [junit4]   2> 		at org.apache.solr.handler.loader.XMLLoader.load(XMLLoader.java:177)
   [junit4]   2> 		at org.apache.solr.handler.UpdateRequestHandler$1.load(UpdateRequestHandler.java:98)
   [junit4]   2> 		at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:74)
   [junit4]   2> 		at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:143)
   [junit4]   2> 		at org.apache.solr.core.SolrCore.execute(SolrCore.java:2051)
   [junit4]   2> 		at org.apache.solr.servlet.DirectSolrConnection.request(DirectSolrConnection.java:131)
   [junit4]   2> 		at org.apache.solr.util.TestHarness.update(TestHarness.java:267)
   [junit4]   2> 		at org.apache.solr.util.BaseTestHarness.checkUpdateStatus(BaseTestHarness.java:282)
   [junit4]   2> 		at org.apache.solr.util.BaseTestHarness.validateUpdate(BaseTestHarness.java:252)
   [junit4]   2> 		at org.apache.solr.SolrTestCaseJ4.checkUpdateU(SolrTestCaseJ4.java:748)
   [junit4]   2> 		at org.apache.solr.SolrTestCaseJ4.assertU(SolrTestCaseJ4.java:727)
   [junit4]   2> 		at org.apache.solr.SolrTestCaseJ4.assertU(SolrTestCaseJ4.java:721)
   [junit4]   2> 		at org.apache.solr.ConvertedLegacyTest.testABunchOfConvertedStuff(ConvertedLegacyTest.java:51)
   [junit4]   2> 		at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   2> 		at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   [junit4]   2> 		at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2> 		at java.lang.reflect.Method.invoke(Method.java:497)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2> 		at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
   [junit4]   2> 		at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2> 		at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
   [junit4]   2> 		at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2> 		at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2> 		at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 		at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 		at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   2> 		at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2> 		at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2> 		at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   2> 		at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 		Locked synchronizers:
   [junit4]   2> 		- java.util.concurrent.locks.ReentrantLock$FairSync@38d66f01
   [junit4]   2> 		- java.util.concurrent.locks.ReentrantLock$NonfairSync@2504e3c6
   [junit4]   2> 	
   [junit4]   2> 	"SUITE-ConvertedLegacyTest-seed#[9A942A4156E3F0E8]" ID=5224 RUNNABLE
   [junit4]   2> 		at sun.management.ThreadImpl.dumpThreads0(Native Method)
   [junit4]   2> 		at sun.management.ThreadImpl.dumpAllThreads(ThreadImpl.java:446)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.ThreadLeakControl.formatThreadStacksFull(ThreadLeakControl.java:671)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.ThreadLeakControl.access$900(ThreadLeakControl.java:62)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.ThreadLeakControl$2.evaluate(ThreadLeakControl.java:412)
   [junit4]   2> 		- locked java.lang.Object@28773b64
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:651)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.RandomizedRunner.access$200(RandomizedRunner.java:138)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.RandomizedRunner$1.run(RandomizedRunner.java:568)
   [junit4]   2> 	
   [junit4]   2> 	"Thread-675" ID=2825 TIMED_WAITING
   [junit4]   2> 		at java.lang.Thread.sleep(Native Method)
   [junit4]   2> 		at org.apache.zookeeper.Login$1.run(Login.java:183)
   [junit4]   2> 		at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 	
   [junit4]   2> 	"JUnit4-serializer-daemon" ID=10 TIMED_WAITING
   [junit4]   2> 		at java.lang.Thread.sleep(Native Method)
   [junit4]   2> 		at com.carrotsearch.ant.tasks.junit4.events.Serializer$1.run(Serializer.java:47)
   [junit4]   2> 	
   [junit4]   2> 	"Attach Listener" ID=5 RUNNABLE
   [junit4]   2> 	
   [junit4]   2> 	"Signal Dispatcher" ID=4 RUNNABLE
   [junit4]   2> 	
   [junit4]   2> 	"Finalizer" ID=3 WAITING on java.lang.ref.ReferenceQueue$Lock@23d2e70c
   [junit4]   2> 		at java.lang.Object.wait(Native Method)
   [junit4]   2> 		- waiting on java.lang.ref.ReferenceQueue$Lock@23d2e70c
   [junit4]   2> 		at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:143)
   [junit4]   2> 		at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:164)
   [junit4]   2> 		at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:209)
   [junit4]   2> 	
   [junit4]   2> 	"Reference Handler" ID=2 WAITING on java.lang.ref.Reference$Lock@214a3a63
   [junit4]   2> 		at java.lang.Object.wait(Native Method)
   [junit4]   2> 		- waiting on java.lang.ref.Reference$Lock@214a3a63
   [junit4]   2> 		at java.lang.Object.wait(Object.java:502)
   [junit4]   2> 		at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:157)
   [junit4]   2> 	
   [junit4]   2> 	"main" ID=1 WAITING on com.carrotsearch.randomizedtesting.RandomizedRunner$1@41616b2
   [junit4]   2> 		at java.lang.Object.wait(Native Method)
   [junit4]   2> 		- waiting on com.carrotsearch.randomizedtesting.RandomizedRunner$1@41616b2
   [junit4]   2> 		at java.lang.Thread.join(Thread.java:1245)
   [junit4]   2> 		at java.lang.Thread.join(Thread.java:1319)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:578)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.RandomizedRunner.run(RandomizedRunner.java:444)
   [junit4]   2> 		at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.execute(SlaveMain.java:199)
   [junit4]   2> 		at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.main(SlaveMain.java:310)
   [junit4]   2> 		at com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe.main(SlaveMainSafe.java:12)
   [junit4]   2> 	
   [junit4]   2> 	^^==============================================
   [junit4]   2> 	
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=ConvertedLegacyTest -Dtests.method=testABunchOfConvertedStuff -Dtests.seed=9A942A4156E3F0E8 -Dtests.slow=true -Dtests.locale=es_EC -Dtests.timezone=America/Atikokan -Dtests.asserts=true -Dtests.file.encoding=Cp1252
   [junit4] ERROR   7245s J1 | ConvertedLegacyTest.testABunchOfConvertedStuff <<<
   [junit4]    > Throwable #1: java.lang.Exception: Test abandoned because suite timeout was reached.
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([9A942A4156E3F0E8]:0)
   [junit4]   2> 7246124 T5224 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 3 leaked thread(s).
   [junit4]   2> 7266343 T5224 ccr.ThreadLeakControl.checkThreadLeaks SEVERE 3 threads leaked from SUITE scope at org.apache.solr.ConvertedLegacyTest: 
   [junit4]   2> 	   1) Thread[id=5225, name=TEST-ConvertedLegacyTest.testABunchOfConvertedStuff-seed#[9A942A4156E3F0E8], state=WAITING, group=TGRP-ConvertedLegacyTest]
   [junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
   [junit4]   2> 	        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
   [junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
   [junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870)
   [junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199)
   [junit4]   2> 	        at java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.lock(ReentrantReadWriteLock.java:943)
   [junit4]   2> 	        at org.apache.solr.update.VersionInfo.blockUpdates(VersionInfo.java:119)
   [junit4]   2> 	        at org.apache.solr.update.UpdateLog.onFirstSearcher(UpdateLog.java:1604)
   [junit4]   2> 	        at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1810)
   [junit4]   2> 	        at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1505)
   [junit4]   2> 	        at org.apache.solr.update.DirectUpdateHandler2.commit(DirectUpdateHandler2.java:617)
   [junit4]   2> 	        at org.apache.solr.update.processor.RunUpdateProcessor.processCommit(RunUpdateProcessorFactory.java:95)
   [junit4]   2> 	        at org.apache.solr.update.processor.UpdateRequestProcessor.processCommit(UpdateRequestProcessor.java:64)
   [junit4]   2> 	        at org.apache.solr.update.processor.DistributedUpdateProcessor.doLocalCommit(DistributedUpdateProcessor.java:1642)
   [junit4]   2> 	        at org.apache.solr.update.processor.DistributedUpdateProcessor.processCommit(DistributedUpdateProcessor.java:1619)
   [junit4]   2> 	        at org.apache.solr.update.processor.LogUpdateProcessor.processCommit(LogUpdateProcessorFactory.java:161)
   [junit4]   2> 	        at org.apache.solr.handler.loader.XMLLoader.processUpdate(XMLLoader.java:270)
   [junit4]   2> 	        at org.apache.solr.handler.loader.XMLLoader.load(XMLLoader.java:177)
   [junit4]   2> 	        at org.apache.solr.handler.UpdateRequestHandler$1.load(UpdateRequestHandler.java:98)
   [junit4]   2> 	        at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:74)
   [junit4]   2> 	        at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:143)
   [junit4]   2> 	        at org.apache.solr.core.SolrCore.execute(SolrCore.java:2051)
   [junit4]   2> 	        at org.apache.solr.servlet.DirectSolrConnection.request(DirectSolrConnection.java:131)
   [junit4]   2> 	        at org.apache.solr.util.TestHarness.update(TestHarness.java:267)
   [junit4]   2> 	        at org.apache.solr.util.BaseTestHarness.checkUpdateStatus(BaseTestHarness.java:282)
   [junit4]   2> 	        at org.apache.solr.util.BaseTestHarness.validateUpdate(BaseTestHarness.java:252)
   [junit4]   2> 	        at org.apache.solr.SolrTestCaseJ4.checkUpdateU(SolrTestCaseJ4.java:748)
   [junit4]   2> 	        at org.apache.solr.SolrTestCaseJ4.assertU(SolrTestCaseJ4.java:727)
   [junit4]   2> 	        at org.apache.solr.SolrTestCaseJ4.assertU(SolrTestCaseJ4.java:721)
   [junit4]   2> 	        at org.apache.solr.ConvertedLegacyTest.testABunchOfConvertedStuff(ConvertedLegacyTest.java:51)
   [junit4]   2> 	        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   2> 	        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   [junit4]   2> 	        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2> 	        at java.lang.reflect.Method.invoke(Method.java:497)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2> 	        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
   [junit4]   2> 	        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2> 	        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
   [junit4]   2> 	        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2> 	        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2> 	        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   2> 	        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2> 	        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2> 	        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   2> 	        at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 	   2) Thread[id=5228, name=Thread-1383, state=WAITING, group=TGRP-ConvertedLegacyTest]
   [junit4]   2> 	        at java.lang.Object.wait(Native Method)
   [junit4]   2> 	        at java.lang.Object.wait(Object.java:502)
   [junit4]   2> 	        at org.apache.solr.core.CloserThread.run(CoreContainer.java:1035)
   [junit4]   2> 	   3) Thread[id=5227, name=searcherExecutor-2318-thread-1, state=WAITING, group=TGRP-ConvertedLegacyTest]
   [junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
   [junit4]   2> 	        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
   [junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
   [junit4]   2> 	        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
   [junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]   2> 	        at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 7266343 T5224 ccr.ThreadLeakControl.tryToInterruptAll Starting to interrupt leaked threads:
   [junit4]   2> 	   1) Thread[id=5225, name=TEST-ConvertedLegacyTest.testABunchOfConvertedStuff-seed#[9A942A4156E3F0E8], state=WAITING, group=TGRP-ConvertedLegacyTest]
   [junit4]   2> 	   2) Thread[id=5228, name=Thread-1383, state=WAITING, group=TGRP-ConvertedLegacyTest]
   [junit4]   2> 	   3) Thread[id=5227, name=searcherExecutor-2318-thread-1, state=WAITING, group=TGRP-ConvertedLegacyTest]
   [junit4]   2> 7269383 T5224 ccr.ThreadLeakControl.tryToInterruptAll SEVERE There are still zombie threads that couldn't be terminated:
   [junit4]   2> 	   1) Thread[id=5225, name=TEST-ConvertedLegacyTest.testABunchOfConvertedStuff-seed#[9A942A4156E3F0E8], state=WAITING, group=TGRP-ConvertedLegacyTest]
   [junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
   [junit4]   2> 	        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
   [junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
   [junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870)
   [junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199)
   [junit4]   2> 	        at java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.lock(ReentrantReadWriteLock.java:943)
   [junit4]   2> 	        at org.apache.solr.update.VersionInfo.blockUpdates(VersionInfo.java:119)
   [junit4]   2> 	        at org.apache.solr.update.UpdateLog.onFirstSearcher(UpdateLog.java:1604)
   [junit4]   2> 	        at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1810)
   [junit4]   2> 	        at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1505)
   [junit4]   2> 	        at org.apache.solr.update.DirectUpdateHandler2.commit(DirectUpdateHandler2.java:617)
   [junit4]   2> 	        at org.apache.solr.update.processor.RunUpdateProcessor.processCommit(RunUpdateProcessorFactory.java:95)
   [junit4]   2> 	        at org.apache.solr.update.processor.UpdateRequestProcessor.processCommit(UpdateRequestProcessor.java:64)
   [junit4]   2> 	        at org.apache.solr.update.processor.DistributedUpdateProcessor.doLocalCommit(DistributedUpdateProcessor.java:1642)
   [junit4]   2> 	        at org.apache.solr.update.processor.DistributedUpdateProcessor.processCommit(DistributedUpdateProcessor.java:1619)
   [junit4]   2> 	        at org.apache.solr.update.processor.LogUpdateProcessor.processCommit(LogUpdateProcessorFactory.java:161)
   [junit4]   2> 	        at org.apache.solr.handler.loader.XMLLoader.processUpdate(XMLLoader.java:270)
   [junit4]   2> 	        at org.apache.solr.handler.loader.XMLLoader.load(XMLLoader.java:177)
   [junit4]   2> 	        at org.apache.solr.handler.UpdateRequestHandler$1.load(UpdateRequestHandler.java:98)
   [junit4]   2> 	        at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:74)
   [junit4]   2> 	        at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:143)
   [junit4]   2> 	        at org.apache.solr.core.SolrCore.execute(SolrCore.java:2051)
   [junit4]   2> 	        at org.apache.solr.servlet.DirectSolrConnection.request(DirectSolrConnection.java:131)
   [junit4]   2> 	        at org.apache.solr.util.TestHarness.update(TestHarness.java:267)
   [junit4]   2> 	        at org.apache.solr.util.BaseTestHarness.checkUpdateStatus(BaseTestHarness.java:282)
   [junit4]   2> 	        at org.apache.solr.util.BaseTestHarness.validateUpdate(BaseTestHarness.java:252)
   [junit4]   2> 	        at org.apache.solr.SolrTestCaseJ4.checkUpdateU(SolrTestCaseJ4.java:748)
   [junit4]   2> 	        at org.apache.solr.SolrTestCaseJ4.assertU(SolrTestCaseJ4.java:727)
   [junit4]   2> 	        at org.apache.solr.SolrTestCaseJ4.assertU(SolrTestCaseJ4.java:721)
   [junit4]   2> 	        at org.apache.solr.ConvertedLegacyTest.testABunchOfConvertedStuff(ConvertedLegacyTest.java:51)
   [junit4]   2> 	        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   2> 	        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   [junit4]   2> 	        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2> 	        at java.lang.reflect.Method.invoke(Method.java:497)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2> 	        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
   [junit4]   2> 	        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2> 	        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
   [junit4]   2> 	        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2> 	        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2> 	        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   2> 	        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2> 	        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2> 	        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   2> 	        at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 	   2) Thread[id=5228, name=Thread-1383, state=WAITING, group=TGRP-ConvertedLegacyTest]
   [junit4]   2> 	        at java.lang.Object.wait(Native Method)
   [junit4]   2> 	        at java.lang.Object.wait(Object.java:502)
   [junit4]   2> 	        at org.apache.solr.core.CloserThread.run(CoreContainer.java:1035)
   [junit4]   2> 	   3) Thread[id=5227, name=searcherExecutor-2318-thread-1, state=WAITING, group=TGRP-ConvertedLegacyTest]
   [junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
   [junit4]   2> 	        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
   [junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
   [junit4]   2> 	        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
   [junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]   2> 	        at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene50): {}, docValues:{}, sim=RandomSimilarityProvider(queryNorm=false,coord=crazy): {}, locale=es_EC, timezone=America/Atikokan
   [junit4]   2> NOTE: Windows 7 6.1 amd64/Oracle Corporation 1.8.0_45 (64-bit)/cpus=3,threads=4,free=42589632,total=282181632
   [junit4]   2> NOTE: All tests run in this JVM: [TestAddFieldRealTimeGet, TestInfoStreamLogging, TestHdfsUpdateLog, DistributedFacetPivotSmallTest, TestSolrCloudWithKerberos, TestFoldingMultitermQuery, TestQuerySenderNoQuery, DirectSolrSpellCheckerTest, TestSolrConfigHandlerCloud, ExternalFileFieldSortTest, TestRandomFaceting, HdfsLockFactoryTest, TestDFRSimilarityFactory, HdfsThreadLeakTest, TestFastOutputStream, BasicZkTest, HdfsDirectoryFactoryTest, CloudMLTQParserTest, QueryEqualityTest, CurrencyFieldXmlFileTest, TestIntervalFaceting, SolrCloudExampleTest, TestConfigReload, TestBulkSchemaConcurrent, PreAnalyzedUpdateProcessorTest, DateFieldTest, HdfsWriteToMultipleCollectionsTest, SaslZkACLProviderTest, BJQParserTest, TestRequestStatusCollectionAPI, HdfsCollectionsAPIDistributedZkTest, HttpPartitionTest, DeleteShardTest, TermsComponentTest, AssignTest, OverseerStatusTest, DistributedQueueTest, TestSolrDeletionPolicy2, PrimitiveFieldTypeTest, TestUtils, ChaosMonkeySafeLeaderTest, TestPHPSerializedResponseWriter, TestFastWriter, PolyFieldTest, TestManagedResourceStorage, TestSimpleQParserPlugin, TestSuggestSpellingConverter, FieldMutatingUpdateProcessorTest, MigrateRouteKeyTest, DistribJoinFromCollectionTest, TestZkChroot, TestValueSourceCache, SignatureUpdateProcessorFactoryTest, TestRecovery, FullSolrCloudDistribCmdsTest, SolrIndexConfigTest, TestInitParams, HdfsRecoverLeaseTest, TestCollapseQParserPlugin, TestSolrDeletionPolicy1, TestConfig, DistributedExpandComponentTest, IndexBasedSpellCheckerTest, ConvertedLegacyTest]
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=ConvertedLegacyTest -Dtests.seed=9A942A4156E3F0E8 -Dtests.slow=true -Dtests.locale=es_EC -Dtests.timezone=America/Atikokan -Dtests.asserts=true -Dtests.file.encoding=Cp1252
   [junit4] ERROR   0.00s J1 | ConvertedLegacyTest (suite) <<<
   [junit4]    > Throwable #1: java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([9A942A4156E3F0E8]:0)
   [junit4] Completed [440/502] on J1 in 7269.42s, 1 test, 2 errors <<< FAILURES!

[...truncated 198 lines...]
BUILD FAILED
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\build.xml:526: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\build.xml:474: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\build.xml:61: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\extra-targets.xml:39: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build.xml:229: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\common-build.xml:512: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\common-build.xml:1415: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\common-build.xml:973: There were test failures: 502 suites, 1983 tests, 1 suite-level error, 1 error, 414 ignored (32 assumptions)

Total time: 151 minutes 54 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-trunk-Windows (32bit/jdk1.8.0_45) - Build # 4844 - Still Failing!

Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Windows/4844/
Java: 32bit/jdk1.8.0_45 -client -XX:+UseG1GC

1 tests failed.
FAILED:  org.apache.solr.cloud.CdcrReplicationHandlerTest.doTest

Error Message:
There are still nodes recoverying - waited for 330 seconds

Stack Trace:
java.lang.AssertionError: There are still nodes recoverying - waited for 330 seconds
	at __randomizedtesting.SeedInfo.seed([60446B9388DD29F2:C700D337E5663A4B]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:172)
	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:133)
	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:128)
	at org.apache.solr.cloud.BaseCdcrDistributedZkTest.waitForRecoveriesToFinish(BaseCdcrDistributedZkTest.java:419)
	at org.apache.solr.cloud.BaseCdcrDistributedZkTest.restartServer(BaseCdcrDistributedZkTest.java:460)
	at org.apache.solr.cloud.CdcrReplicationHandlerTest.doTestPartialReplication(CdcrReplicationHandlerTest.java:109)
	at org.apache.solr.cloud.CdcrReplicationHandlerTest.doTest(CdcrReplicationHandlerTest.java:51)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:497)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:960)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:935)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 10036 lines...]
   [junit4] Suite: org.apache.solr.cloud.CdcrReplicationHandlerTest
   [junit4]   2> Creating dataDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.CdcrReplicationHandlerTest 60446B9388DD29F2-001\init-core-data-001
   [junit4]   2> 1508634 T8146 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (false) and clientAuth (false)
   [junit4]   2> 1508634 T8146 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
   [junit4]   2> 1508637 T8146 oasc.ZkTestServer.run STARTING ZK TEST SERVER
   [junit4]   2> 1508638 T8147 oasc.ZkTestServer$2$1.setClientPort client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1508638 T8147 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
   [junit4]   2> 1508738 T8146 oasc.ZkTestServer.run start zk server on port:59783
   [junit4]   2> 1508738 T8146 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 1508740 T8146 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1508744 T8154 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@beae11 name:ZooKeeperConnection Watcher:127.0.0.1:59783 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1508745 T8146 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1508745 T8146 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 1508745 T8146 oascc.SolrZkClient.makePath makePath: /solr
   [junit4]   2> 1508750 T8146 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 1508753 T8146 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1508755 T8157 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1db19a5 name:ZooKeeperConnection Watcher:127.0.0.1:59783/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1508755 T8146 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1508756 T8146 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 1508756 T8146 oascc.SolrZkClient.makePath makePath: /collections/collection1
   [junit4]   2> 1508759 T8146 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
   [junit4]   2> 1508763 T8146 oascc.SolrZkClient.makePath makePath: /collections/control_collection
   [junit4]   2> 1508766 T8146 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
   [junit4]   2> 1508769 T8146 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\solrconfig-cdcr.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 1508770 T8146 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 1508774 T8146 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\schema15.xml to /configs/conf1/schema.xml
   [junit4]   2> 1508774 T8146 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
   [junit4]   2> 1508778 T8146 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1508779 T8146 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1508782 T8146 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 1508783 T8146 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 1508786 T8146 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 1508787 T8146 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
   [junit4]   2> 1508790 T8146 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 1508791 T8146 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
   [junit4]   2> 1508794 T8146 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 1508794 T8146 oascc.SolrZkClient.makePath makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 1508798 T8146 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1508798 T8146 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1508802 T8146 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1508802 T8146 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1508806 T8146 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 1508806 T8146 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 1508809 T8146 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 1508810 T8146 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 1508814 T8146 oas.SolrTestCaseJ4.setUp ###Starting doTest
   [junit4]   2> 1509107 T8146 oas.SolrTestCaseJ4.writeCoreProperties Writing core.properties file to C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.CdcrReplicationHandlerTest 60446B9388DD29F2-001\jetty-001\cores\collection1
   [junit4]   2> 1509111 T8146 oejs.Server.doStart jetty-9.2.10.v20150310
   [junit4]   2> 1509115 T8146 oejsh.ContextHandler.doStart Started o.e.j.s.ServletContextHandler@1f43cfb{/,null,AVAILABLE}
   [junit4]   2> 1509119 T8146 oejs.AbstractConnector.doStart Started ServerConnector@13b032{HTTP/1.1}{127.0.0.1:59790}
   [junit4]   2> 1509120 T8146 oejs.Server.doStart Started @1509595ms
   [junit4]   2> 1509120 T8146 oascse.JettySolrRunner$1.lifeCycleStarted Jetty properties: {shards=shard1, hostContext=/, hostPort=59790, coreRootDirectory=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.CdcrReplicationHandlerTest 60446B9388DD29F2-001\jetty-001\cores}
   [junit4]   2> 1509120 T8146 oass.SolrDispatchFilter.init SolrDispatchFilter.init()sun.misc.Launcher$AppClassLoader@e2f2a
   [junit4]   2> 1509121 T8146 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.CdcrReplicationHandlerTest 60446B9388DD29F2-001\jetty-001\'
   [junit4]   2> 1509149 T8146 oasc.SolrXmlConfig.fromFile Loading container configuration from C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.CdcrReplicationHandlerTest 60446B9388DD29F2-001\jetty-001\solr.xml
   [junit4]   2> 1509157 T8146 oasc.CorePropertiesLocator.<init> Config-defined core root directory: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.CdcrReplicationHandlerTest 60446B9388DD29F2-001\jetty-001\cores
   [junit4]   2> 1509157 T8146 oasc.CoreContainer.<init> New CoreContainer 23221969
   [junit4]   2> 1509158 T8146 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.CdcrReplicationHandlerTest 60446B9388DD29F2-001\jetty-001\]
   [junit4]   2> 1509158 T8146 oasc.CoreContainer.load loading shared library: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.CdcrReplicationHandlerTest 60446B9388DD29F2-001\jetty-001\lib
   [junit4]   2> 1509158 T8146 oasc.SolrResourceLoader.addToClassLoader WARN Can't find (or read) directory to add to classloader: lib (resolved as: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.CdcrReplicationHandlerTest 60446B9388DD29F2-001\jetty-001\lib).
   [junit4]   2> 1509167 T8146 oashc.HttpShardHandlerFactory.init created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1509170 T8146 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1509171 T8146 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1509171 T8146 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1509172 T8146 oasc.CoreContainer.load Node Name: 127.0.0.1
   [junit4]   2> 1509172 T8146 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:59783/solr
   [junit4]   2> 1509172 T8146 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 1509173 T8146 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 1509174 T8146 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1509176 T8171 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5628f7 name:ZooKeeperConnection Watcher:127.0.0.1:59783 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1509176 T8146 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1509177 T8146 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 1509180 T8146 n:127.0.0.1:59790_ oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1509182 T8174 n:127.0.0.1:59790_ oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1627508 name:ZooKeeperConnection Watcher:127.0.0.1:59783/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1509182 T8146 n:127.0.0.1:59790_ oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1509184 T8146 n:127.0.0.1:59790_ oascc.SolrZkClient.makePath makePath: /overseer/queue
   [junit4]   2> 1509188 T8146 n:127.0.0.1:59790_ oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
   [junit4]   2> 1509191 T8146 n:127.0.0.1:59790_ oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
   [junit4]   2> 1509195 T8146 n:127.0.0.1:59790_ oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
   [junit4]   2> 1509199 T8146 n:127.0.0.1:59790_ oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
   [junit4]   2> 1509203 T8146 n:127.0.0.1:59790_ oascc.SolrZkClient.makePath makePath: /live_nodes
   [junit4]   2> 1509207 T8146 n:127.0.0.1:59790_ oascc.SolrZkClient.makePath makePath: /aliases.json
   [junit4]   2> 1509210 T8146 n:127.0.0.1:59790_ oascc.SolrZkClient.makePath makePath: /clusterstate.json
   [junit4]   2> 1509212 T8146 n:127.0.0.1:59790_ oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:59790_
   [junit4]   2> 1509213 T8146 n:127.0.0.1:59790_ oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:59790_
   [junit4]   2> 1509216 T8146 n:127.0.0.1:59790_ oascc.SolrZkClient.makePath makePath: /overseer_elect
   [junit4]   2> 1509219 T8146 n:127.0.0.1:59790_ oascc.SolrZkClient.makePath makePath: /overseer_elect/election
   [junit4]   2> 1509221 T8146 n:127.0.0.1:59790_ oasc.Overseer.close Overseer (id=null) closing
   [junit4]   2> 1509224 T8146 n:127.0.0.1:59790_ oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:59790_
   [junit4]   2> 1509224 T8146 n:127.0.0.1:59790_ oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
   [junit4]   2> 1509227 T8146 n:127.0.0.1:59790_ oasc.Overseer.start Overseer (id=93872530158583811-127.0.0.1:59790_-n_0000000000) starting
   [junit4]   2> 1509230 T8146 n:127.0.0.1:59790_ oascc.SolrZkClient.makePath makePath: /overseer/queue-work
   [junit4]   2> 1509241 T8146 n:127.0.0.1:59790_ oasc.OverseerAutoReplicaFailoverThread.<init> Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 1509242 T8176 n:127.0.0.1:59790_ oasc.OverseerCollectionProcessor.run Process current queue of collection creations
   [junit4]   2> 1509242 T8146 n:127.0.0.1:59790_ oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 1509243 T8175 n:127.0.0.1:59790_ oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
   [junit4]   2> 1509248 T8146 n:127.0.0.1:59790_ oasc.CoreContainer.initializeAuthenticationPlugin No authentication plugin used.
   [junit4]   2> 1509249 T8146 n:127.0.0.1:59790_ oasc.CoreContainer.intializeAuthorizationPlugin Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1509251 T8146 n:127.0.0.1:59790_ oasc.CorePropertiesLocator.discover Looking for core definitions underneath C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.CdcrReplicationHandlerTest 60446B9388DD29F2-001\jetty-001\cores
   [junit4]   2> 1509252 T8146 n:127.0.0.1:59790_ oasc.CoreDescriptor.<init> CORE DESCRIPTOR: {name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, loadOnStartup=true, instanceDir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.CdcrReplicationHandlerTest 60446B9388DD29F2-001\jetty-001\cores\collection1, collection=tmp_collection, absoluteInstDir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.CdcrReplicationHandlerTest 60446B9388DD29F2-001\jetty-001\cores\collection1\, coreNodeName=, dataDir=data\, shard=}
   [junit4]   2> 1509253 T8146 n:127.0.0.1:59790_ oasc.CorePropertiesLocator.discoverUnder Found core collection1 in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.CdcrReplicationHandlerTest 60446B9388DD29F2-001\jetty-001\cores\collection1\
   [junit4]   2> 1509253 T8146 n:127.0.0.1:59790_ oasc.CorePropertiesLocator.discover Found 1 core definitions
   [junit4]   2> 1509255 T8178 n:127.0.0.1:59790_ c:tmp_collection x:collection1 oasc.ZkController.publish publishing core=collection1 state=down collection=tmp_collection
   [junit4]   2> 1509255 T8178 n:127.0.0.1:59790_ c:tmp_collection x:collection1 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 1509256 T8174 n:127.0.0.1:59790_ oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 1509256 T8178 n:127.0.0.1:59790_ oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 1509258 T8175 n:127.0.0.1:59790_ oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"http://127.0.0.1:59790",
   [junit4]   2> 	  "node_name":"127.0.0.1:59790_",
   [junit4]   2> 	  "numShards":"1",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "collection":"tmp_collection",
   [junit4]   2> 	  "operation":"state"} current state version: 0
   [junit4]   2> 1509258 T8175 n:127.0.0.1:59790_ oasco.ReplicaMutator.updateState Update state numShards=1 message={
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"http://127.0.0.1:59790",
   [junit4]   2> 	  "node_name":"127.0.0.1:59790_",
   [junit4]   2> 	  "numShards":"1",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "collection":"tmp_collection",
   [junit4]   2> 	  "operation":"state"}
   [junit4]   2> 1509259 T8175 n:127.0.0.1:59790_ oasco.ClusterStateMutator.createCollection building a new cName: tmp_collection
   [junit4]   2> 1509259 T8175 n:127.0.0.1:59790_ oasco.ReplicaMutator.updateState Assigning new node to shard shard=shard1
   [junit4]   2> 1509261 T8174 n:127.0.0.1:59790_ oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1510257 T8178 n:127.0.0.1:59790_ oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1510257 T8178 n:127.0.0.1:59790_ oasc.ZkController.createCollectionZkNode Check for collection zkNode:tmp_collection
   [junit4]   2> 1510258 T8178 n:127.0.0.1:59790_ oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:tmp_collection
   [junit4]   2> 1510258 T8178 n:127.0.0.1:59790_ oasc.ZkController.getConfName Looking for collection configName
   [junit4]   2> 1510259 T8178 n:127.0.0.1:59790_ oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
   [junit4]   2> 1510259 T8178 n:127.0.0.1:59790_ oascc.SolrZkClient.makePath makePath: /collections/tmp_collection
   [junit4]   2> 1510262 T8178 n:127.0.0.1:59790_ oascc.ZkStateReader.readConfigName Load collection config from:/collections/tmp_collection
   [junit4]   2> 1510263 T8178 n:127.0.0.1:59790_ oascc.ZkStateReader.readConfigName path=/collections/tmp_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1510263 T8178 n:127.0.0.1:59790_ oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.CdcrReplicationHandlerTest 60446B9388DD29F2-001\jetty-001\cores\collection1\'
   [junit4]   2> 1510283 T8178 n:127.0.0.1:59790_ oasc.Config.<init> loaded config solrconfig.xml with version 0 
   [junit4]   2> 1510291 T8178 n:127.0.0.1:59790_ oasc.SolrConfig.refreshRequestParams current version of requestparams : -1
   [junit4]   2> 1510294 T8178 n:127.0.0.1:59790_ oasc.SolrConfig.<init> Using Lucene MatchVersion: 6.0.0
   [junit4]   2> 1510301 T8178 n:127.0.0.1:59790_ oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1510303 T8178 n:127.0.0.1:59790_ oass.IndexSchema.readSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 1510308 T8178 n:127.0.0.1:59790_ oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 1510394 T8178 n:127.0.0.1:59790_ oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 1510395 T8178 n:127.0.0.1:59790_ oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 1510395 T8178 n:127.0.0.1:59790_ oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 1510398 T8178 n:127.0.0.1:59790_ oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 1510426 T8178 n:127.0.0.1:59790_ oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection tmp_collection
   [junit4]   2> 1510426 T8178 n:127.0.0.1:59790_ x:collection1 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
   [junit4]   2> 1510427 T8178 n:127.0.0.1:59790_ x:collection1 oasc.SolrCore.<init> [[collection1] ] Opening new SolrCore at [C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.CdcrReplicationHandlerTest 60446B9388DD29F2-001\jetty-001\cores\collection1\], dataDir=[null]
   [junit4]   2> 1510427 T8178 n:127.0.0.1:59790_ x:collection1 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@15bd0d5
   [junit4]   2> 1510428 T8178 n:127.0.0.1:59790_ x:collection1 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.CdcrReplicationHandlerTest 60446B9388DD29F2-001\jetty-001\cores\collection1\data
   [junit4]   2> 1510429 T8178 n:127.0.0.1:59790_ x:collection1 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.CdcrReplicationHandlerTest 60446B9388DD29F2-001\jetty-001\cores\collection1\data\index/
   [junit4]   2> 1510430 T8178 n:127.0.0.1:59790_ x:collection1 oasc.SolrCore.initIndex WARN [collection1] Solr index directory 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.CdcrReplicationHandlerTest 60446B9388DD29F2-001\jetty-001\cores\collection1\data\index' doesn't exist. Creating new index...
   [junit4]   2> 1510431 T8178 n:127.0.0.1:59790_ x:collection1 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.CdcrReplicationHandlerTest 60446B9388DD29F2-001\jetty-001\cores\collection1\data\index
   [junit4]   2> 1510431 T8178 n:127.0.0.1:59790_ x:collection1 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=34, maxMergeAtOnceExplicit=43, maxMergedSegmentMB=55.6396484375, floorSegmentMB=1.296875, forceMergeDeletesPctAllowed=22.934645914520658, segmentsPerTier=27.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.20246057691364927
   [junit4]   2> 1510482 T8178 n:127.0.0.1:59790_ x:collection1 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.CdcrReplicationHandlerTest 60446B9388DD29F2-001\jetty-001\cores\collection1\data\index,segFN=segments_1,generation=1}
   [junit4]   2> 1510482 T8178 n:127.0.0.1:59790_ x:collection1 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 1510488 T8178 n:127.0.0.1:59790_ x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "cdcr-processor-chain"
   [junit4]   2> 1510488 T8178 n:127.0.0.1:59790_ x:collection1 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 1510493 T8178 n:127.0.0.1:59790_ x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 1510495 T8178 n:127.0.0.1:59790_ x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 1510497 T8178 n:127.0.0.1:59790_ x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 1510498 T8178 n:127.0.0.1:59790_ x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 1510499 T8178 n:127.0.0.1:59790_ x:collection1 oasc.PluginBag.createPlugin requestHandler : '/replication' created with startup=lazy 
   [junit4]   2> 1510506 T8178 n:127.0.0.1:59790_ x:collection1 oasc.SolrResourceLoader.findClass WARN Solr loaded a deprecated plugin/analysis class [org.apache.solr.handler.admin.AdminHandlers]. Please consult documentation how to replace it accordingly.
   [junit4]   2> 1510507 T8178 n:127.0.0.1:59790_ x:collection1 oasc.SolrResourceLoader.findClass WARN Solr loaded a deprecated plugin/analysis class [org.apache.solr.handler.admin.AdminHandlers]. Please consult documentation how to replace it accordingly.
   [junit4]   2> 1510507 T8178 n:127.0.0.1:59790_ x:collection1 oasc.RequestHandlers.initHandlersFromConfig Registered paths: /admin/mbeans,standard,/update/csv,/update/json/docs,/admin/luke,/admin/segments,/get,/admin/system,/replication,/admin/properties,/config,/schema,/admin/plugins,/admin/logging,/update/json,/admin,/admin/threads,/admin/ping,/cdcr,/update,/admin/file
   [junit4]   2> 1510507 T8178 n:127.0.0.1:59790_ x:collection1 oasc.SolrCore.initStatsCache Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 1510508 T8178 n:127.0.0.1:59790_ x:collection1 oasu.UpdateHandler.<init> Using UpdateLog implementation: org.apache.solr.update.CdcrUpdateLog
   [junit4]   2> 1510508 T8178 n:127.0.0.1:59790_ x:collection1 oasu.UpdateLog.init Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1510510 T8178 n:127.0.0.1:59790_ x:collection1 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 1510510 T8178 n:127.0.0.1:59790_ x:collection1 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 1510511 T8178 n:127.0.0.1:59790_ x:collection1 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=43, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
   [junit4]   2> 1510512 T8178 n:127.0.0.1:59790_ x:collection1 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.CdcrReplicationHandlerTest 60446B9388DD29F2-001\jetty-001\cores\collection1\data\index,segFN=segments_1,generation=1}
   [junit4]   2> 1510512 T8178 n:127.0.0.1:59790_ x:collection1 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 1510513 T8178 n:127.0.0.1:59790_ x:collection1 oass.SolrIndexSearcher.<init> Opening Searcher@301589[collection1] main
   [junit4]   2> 1510513 T8178 n:127.0.0.1:59790_ x:collection1 oasu.UpdateLog.onFirstSearcher On first searcher opened, looking up max value of version field
   [junit4]   2> 1510513 T8178 n:127.0.0.1:59790_ x:collection1 oasu.VersionInfo.getMaxVersionFromIndex Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 1510513 T8178 n:127.0.0.1:59790_ x:collection1 oasu.VersionInfo.getMaxVersionFromIndex WARN No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 1510514 T8178 n:127.0.0.1:59790_ x:collection1 oasu.UpdateLog.seedBucketsWithHighestVersion WARN Could not find max version in index or recent updates, using new clock 1501960484499226624
   [junit4]   2> 1510515 T8178 n:127.0.0.1:59790_ x:collection1 oasu.UpdateLog.seedBucketsWithHighestVersion Took 2 ms to seed version buckets with highest version 1501960484499226624
   [junit4]   2> 1510515 T8178 n:127.0.0.1:59790_ x:collection1 oascc.ZkStateReader.readConfigName Load collection config from:/collections/tmp_collection
   [junit4]   2> 1510518 T8178 n:127.0.0.1:59790_ x:collection1 oascc.ZkStateReader.readConfigName path=/collections/tmp_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1510518 T8178 n:127.0.0.1:59790_ x:collection1 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 1510518 T8178 n:127.0.0.1:59790_ x:collection1 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 1510519 T8178 n:127.0.0.1:59790_ x:collection1 oasr.RestManager.init Initializing RestManager with initArgs: {}
   [junit4]   2> 1510519 T8178 n:127.0.0.1:59790_ x:collection1 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1510520 T8178 n:127.0.0.1:59790_ x:collection1 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 1510520 T8178 n:127.0.0.1:59790_ x:collection1 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1510520 T8178 n:127.0.0.1:59790_ x:collection1 oasr.RestManager.init Initializing 0 registered ManagedResources
   [junit4]   2> 1510521 T8178 n:127.0.0.1:59790_ x:collection1 oascc.SolrZkClient.makePath makePath: /collections/tmp_collection/cdcr/state
   [junit4]   2> 1510525 T8178 n:127.0.0.1:59790_ x:collection1 oash.CdcrBufferStateManager.createStateNode Created znode /collections/tmp_collection/cdcr/state/buffer
   [junit4]   2> 1510527 T8178 n:127.0.0.1:59790_ x:collection1 oash.CdcrProcessStateManager.createStateNode Created znode /collections/tmp_collection/cdcr/state/process
   [junit4]   2> 1510530 T8178 n:127.0.0.1:59790_ x:collection1 oasha.AdminHandlers.inform WARN <requestHandler name="/admin/" 
   [junit4]   2> 	 class="solr.admin.AdminHandlers" /> is deprecated . It is not required anymore
   [junit4]   2> 1510531 T8178 n:127.0.0.1:59790_ x:collection1 oasc.ZkController.getConfDirListeners watch zkdir /configs/conf1
   [junit4]   2> 1510531 T8179 n:127.0.0.1:59790_ x:collection1 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@301589[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1510531 T8178 n:127.0.0.1:59790_ x:collection1 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 1510532 T8183 n:127.0.0.1:59790_ c:tmp_collection s:shard1 x:collection1 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:59790 collection:tmp_collection shard:shard1
   [junit4]   2> 1510533 T8146 n:127.0.0.1:59790_ oass.SolrDispatchFilter.init user.dir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0
   [junit4]   2> 1510533 T8146 n:127.0.0.1:59790_ oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 1510534 T8183 n:127.0.0.1:59790_ c:tmp_collection s:shard1 x:collection1 oascc.SolrZkClient.makePath makePath: /collections/tmp_collection/leader_elect/shard1/election
   [junit4]   2> 1510541 T8183 n:127.0.0.1:59790_ c:tmp_collection s:shard1 x:collection1 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
   [junit4]   2> 1510543 T8174 n:127.0.0.1:59790_ oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 1510544 T8183 n:127.0.0.1:59790_ c:tmp_collection s:shard1 x:collection1 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
   [junit4]   2> 1510544 T8183 n:127.0.0.1:59790_ c:tmp_collection s:shard1 x:collection1 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
   [junit4]   2> ASYNC  NEW_CORE C2050 name=collection1 org.apache.solr.core.SolrCore@123da0a url=http://127.0.0.1:59790/collection1 node=127.0.0.1:59790_ C2050_STATE=coll:tmp_collection core:collection1 props:{core=collection1, base_url=http://127.0.0.1:59790, node_name=127.0.0.1:59790_, state=down}
   [junit4]   2> 1510544 T8183 n:127.0.0.1:59790_ c:tmp_collection s:shard1 x:collection1 C2050 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:59790/collection1/
   [junit4]   2> 1510544 T8183 n:127.0.0.1:59790_ c:tmp_collection s:shard1 x:collection1 C2050 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
   [junit4]   2> 1510545 T8183 n:127.0.0.1:59790_ c:tmp_collection s:shard1 x:collection1 C2050 oasc.SyncStrategy.syncToMe http://127.0.0.1:59790/collection1/ has no replicas
   [junit4]   2> 1510545 T8183 n:127.0.0.1:59790_ c:tmp_collection s:shard1 x:collection1 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:59790/collection1/ shard1
   [junit4]   2> 1510545 T8183 n:127.0.0.1:59790_ c:tmp_collection s:shard1 x:collection1 oascc.SolrZkClient.makePath makePath: /collections/tmp_collection/leaders/shard1
   [junit4]   2> 1510546 T8175 n:127.0.0.1:59790_ oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "operation":"leader",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "collection":"tmp_collection"} current state version: 1
   [junit4]   2> 1510550 T8174 n:127.0.0.1:59790_ oash.CdcrLeaderStateManager$LeaderStateWatcher.process Received new leader state @ tmp_collection:shard1
   [junit4]   2> 1510554 T8174 n:127.0.0.1:59790_ oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 1510555 T8175 n:127.0.0.1:59790_ oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "operation":"leader",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "collection":"tmp_collection",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:59790",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "state":"active"} current state version: 1
   [junit4]   2> 1510662 T8174 n:127.0.0.1:59790_ oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1510707 T8183 n:127.0.0.1:59790_ c:tmp_collection s:shard1 x:collection1 oasc.ZkController.register We are http://127.0.0.1:59790/collection1/ and leader is http://127.0.0.1:59790/collection1/
   [junit4]   2> 1510707 T8183 n:127.0.0.1:59790_ c:tmp_collection s:shard1 x:collection1 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:59790
   [junit4]   2> 1510707 T8183 n:127.0.0.1:59790_ c:tmp_collection s:shard1 x:collection1 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
   [junit4]   2> 1510707 T8183 n:127.0.0.1:59790_ c:tmp_collection s:shard1 x:collection1 oasc.ZkController.publish publishing core=collection1 state=active collection=tmp_collection
   [junit4]   2> 1510708 T8183 n:127.0.0.1:59790_ c:tmp_collection s:shard1 x:collection1 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 1510710 T8174 n:127.0.0.1:59790_ oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 1510712 T8175 n:127.0.0.1:59790_ oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "core_node_name":"core_node1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"http://127.0.0.1:59790",
   [junit4]   2> 	  "node_name":"127.0.0.1:59790_",
   [junit4]   2> 	  "numShards":"1",
   [junit4]   2> 	  "state":"active",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "collection":"tmp_collection",
   [junit4]   2> 	  "operation":"state"} current state version: 2
   [junit4]   2> 1510712 T8175 n:127.0.0.1:59790_ oasco.ReplicaMutator.updateState Update state numShards=1 message={
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "core_node_name":"core_node1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"http://127.0.0.1:59790",
   [junit4]   2> 	  "node_name":"127.0.0.1:59790_",
   [junit4]   2> 	  "numShards":"1",
   [junit4]   2> 	  "state":"active",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "collection":"tmp_collection",
   [junit4]   2> 	  "operation":"state"}
   [junit4]   2> 1510817 T8174 n:127.0.0.1:59790_ oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1510833 T8146 oas.SolrTestCaseJ4.writeCoreProperties Writing core.properties file to C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.CdcrReplicationHandlerTest 60446B9388DD29F2-001\jetty-002\cores\collection1
   [junit4]   2> 1510837 T8146 oejs.Server.doStart jetty-9.2.10.v20150310
   [junit4]   2> 1510841 T8146 oejsh.ContextHandler.doStart Started o.e.j.s.ServletContextHandler@14a7872{/,null,AVAILABLE}
   [junit4]   2> 1510842 T8146 oejs.AbstractConnector.doStart Started ServerConnector@831ddf{HTTP/1.1}{127.0.0.1:59799}
   [junit4]   2> 1510843 T8146 oejs.Server.doStart Started @1511318ms
   [junit4]   2> 1510843 T8146 oascse.JettySolrRunner$1.lifeCycleStarted Jetty properties: {shards=shard2, hostContext=/, hostPort=59799, coreRootDirectory=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.CdcrReplicationHandlerTest 60446B9388DD29F2-001\jetty-002\cores}
   [junit4]   2> 1510843 T8146 oass.SolrDispatchFilter.init SolrDispatchFilter.init()sun.misc.Launcher$AppClassLoader@e2f2a
   [junit4]   2> 1510844 T8146 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.CdcrReplicationHandlerTest 60446B9388DD29F2-001\jetty-002\'
   [junit4]   2> 1510872 T8146 oasc.SolrXmlConfig.fromFile Loading container configuration from C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.CdcrReplicationHandlerTest 60446B9388DD29F2-001\jetty-002\solr.xml
   [junit4]   2> 1510880 T8146 oasc.CorePropertiesLocator.<init> Config-defined core root directory: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.CdcrReplicationHandlerTest 60446B9388DD29F2-001\jetty-002\cores
   [junit4]   2> 1510881 T8146 oasc.CoreContainer.<init> New CoreContainer 16773446
   [junit4]   2> 1510881 T8146 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.CdcrReplicationHandlerTest 60446B9388DD29F2-001\jetty-002\]
   [junit4]   2> 1510882 T8146 oasc.CoreContainer.load loading shared library: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.CdcrReplicationHandlerTest 60446B9388DD29F2-001\jetty-002\lib
   [junit4]   2> 1510882 T8146 oasc.SolrResourceLoader.addToClassLoader WARN Can't find (or read) directory to add to classloader: lib (resolved as: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.CdcrReplicationHandlerTest 60446B9388DD29F2-001\jetty-002\lib).
   [junit4]   2> 1510891 T8146 oashc.HttpShardHandlerFactory.init created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1510894 T8146 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1510895 T8146 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1510895 T8146 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1510895 T8146 oasc.CoreContainer.load Node Name: 127.0.0.1
   [junit4]   2> 1510896 T8146 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:59783/solr
   [junit4]   2> 1510896 T8146 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 1510896 T8146 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 1510897 T8146 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1510899 T8197 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5dcbbe name:ZooKeeperConnection Watcher:127.0.0.1:59783 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1510899 T8146 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1510900 T8146 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 1510903 T8146 n:127.0.0.1:59799_ oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1510905 T8200 n:127.0.0.1:59799_ oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@782bcf name:ZooKeeperConnection Watcher:127.0.0.1:59783/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1510905 T8146 n:127.0.0.1:59799_ oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1510914 T8146 n:127.0.0.1:59799_ oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 1511921 T8146 n:127.0.0.1:59799_ oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:59799_
   [junit4]   2> 1511923 T8146 n:127.0.0.1:59799_ oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:59799_
   [junit4]   2> 1511928 T8146 n:127.0.0.1:59799_ oasc.Overseer.close Overseer (id=null) closing
   [junit4]   2> 1511931 T8146 n:127.0.0.1:59799_ oasc.CoreContainer.initializeAuthenticationPlugin No authentication plugin used.
   [junit4]   2> 1511932 T8146 n:127.0.0.1:59799_ oasc.CoreContainer.intializeAuthorizationPlugin Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1511934 T8146 n:127.0.0.1:59799_ oasc.CorePropertiesLocator.discover Looking for core definitions underneath C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.CdcrReplicationHandlerTest 60446B9388DD29F2-001\jetty-002\cores
   [junit4]   2> 1511935 T8146 n:127.0.0.1:59799_ oasc.CoreDescriptor.<init> CORE DESCRIPTOR: {name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, loadOnStartup=true, instanceDir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.CdcrReplicationHandlerTest 60446B9388DD29F2-001\jetty-002\cores\collection1, collection=tmp_collection, absoluteInstDir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.CdcrReplicationHandlerTest 60446B9388DD29F2-001\jetty-002\cores\collection1\, coreNodeName=, dataDir=data\, shard=}
   [junit4]   2> 1511935 T8146 n:127.0.0.1:59799_ oasc.CorePropertiesLocator.discoverUnder Found core collection1 in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.CdcrReplicationHandlerTest 60446B9388DD29F2-001\jetty-002\cores\collection1\
   [junit4]   2> 1511936 T8146 n:127.0.0.1:59799_ oasc.CorePropertiesLocator.discover Found 1 core definitions
   [junit4]   2> 1511937 T8201 n:127.0.0.1:59799_ c:tmp_collection x:collection1 oasc.ZkController.publish publishing core=collection1 state=down collection=tmp_collection
   [junit4]   2> 1511937 T8201 n:127.0.0.1:59799_ c:tmp_collection x:collection1 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 1511939 T8174 n:127.0.0.1:59790_ oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 1511939 T8201 n:127.0.0.1:59799_ oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 1511939 T8175 n:127.0.0.1:59790_ oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"http://127.0.0.1:59799",
   [junit4]   2> 	  "node_name":"127.0.0.1:59799_",
   [junit4]   2> 	  "numShards":"1",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "collection":"tmp_collection",
   [junit4]   2> 	  "operation":"state"} current state version: 3
   [junit4]   2> 1511941 T8175 n:127.0.0.1:59790_ oasco.ReplicaMutator.updateState Update state numShards=1 message={
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"http://127.0.0.1:59799",
   [junit4]   2> 	  "node_name":"127.0.0.1:59799_",
   [junit4]   2> 	  "numShards":"1",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "collection":"tmp_collection",
   [junit4]   2> 	  "operation":"state"}
   [junit4]   2> 1511941 T8175 n:127.0.0.1:59790_ oasco.ReplicaMutator.updateState Collection already exists with numShards=1
   [junit4]   2> 1511941 T8175 n:127.0.0.1:59790_ oasco.ReplicaMutator.updateState Assigning new node to shard shard=shard1
   [junit4]   2> 1512047 T8200 n:127.0.0.1:59799_ oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1512047 T8174 n:127.0.0.1:59790_ oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1512939 T8201 n:127.0.0.1:59799_ oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1512939 T8201 n:127.0.0.1:59799_ oasc.ZkController.createCollectionZkNode Check for collection zkNode:tmp_collection
   [junit4]   2> 1512939 T8201 n:127.0.0.1:59799_ oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 1512939 T8201 n:127.0.0.1:59799_ oascc.ZkStateReader.readConfigName Load collection config from:/collections/tmp_collection
   [junit4]   2> 1512941 T8201 n:127.0.0.1:59799_ oascc.ZkStateReader.readConfigName path=/collections/tmp_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1512941 T8201 n:127.0.0.1:59799_ oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.CdcrReplicationHandlerTest 60446B9388DD29F2-001\jetty-002\cores\collection1\'
   [junit4]   2> 1512959 T8201 n:127.0.0.1:59799_ oasc.Config.<init> loaded config solrconfig.xml with version 0 
   [junit4]   2> 1512967 T8201 n:127.0.0.1:59799_ oasc.SolrConfig.refreshRequestParams current version of requestparams : -1
   [junit4]   2> 1512971 T8201 n:127.0.0.1:59799_ oasc.SolrConfig.<init> Using Lucene MatchVersion: 6.0.0
   [junit4]   2> 1512978 T8201 n:127.0.0.1:59799_ oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1512980 T8201 n:127.0.0.1:59799_ oass.IndexSchema.readSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 1512984 T8201 n:127.0.0.1:59799_ oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 1513070 T8201 n:127.0.0.1:59799_ oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 1513072 T8201 n:127.0.0.1:59799_ oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 1513073 T8201 n:127.0.0.1:59799_ oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 1513077 T8201 n:127.0.0.1:59799_ oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 1513103 T8201 n:127.0.0.1:59799_ oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection tmp_collection
   [junit4]   2> 1513103 T8201 n:127.0.0.1:59799_ x:collection1 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
   [junit4]   2> 1513104 T8201 n:127.0.0.1:59799_ x:collection1 oasc.SolrCore.<init> [[collection1] ] Opening new SolrCore at [C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.CdcrReplicationHandlerTest 60446B9388DD29F2-001\jetty-002\cores\collection1\], dataDir=[null]
   [junit4]   2> 1513104 T8201 n:127.0.0.1:59799_ x:collection1 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@15bd0d5
   [junit4]   2> 1513105 T8201 n:127.0.0.1:59799_ x:collection1 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.CdcrReplicationHandlerTest 60446B9388DD29F2-001\jetty-002\cores\collection1\data
   [junit4]   2> 1513106 T8201 n:127.0.0.1:59799_ x:collection1 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.CdcrReplicationHandlerTest 60446B9388DD29F2-001\jetty-002\cores\collection1\data\index/
   [junit4]   2> 1513106 T8201 n:127.0.0.1:59799_ x:collection1 oasc.SolrCore.initIndex WARN [collection1] Solr index directory 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.CdcrReplicationHandlerTest 60446B9388DD29F2-001\jetty-002\cores\collection1\data\index' doesn't exist. Creating new index...
   [junit4]   2> 1513107 T8201 n:127.0.0.1:59799_ x:collection1 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.CdcrReplicationHandlerTest 60446B9388DD29F2-001\jetty-002\cores\collection1\data\index
   [junit4]   2> 1513108 T8201 n:127.0.0.1:59799_ x:collection1 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=34, maxMergeAtOnceExplicit=43, maxMergedSegmentMB=55.6396484375, floorSegmentMB=1.296875, forceMergeDeletesPctAllowed=22.934645914520658, segmentsPerTier=27.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.20246057691364927
   [junit4]   2> 1513114 T8201 n:127.0.0.1:59799_ x:collection1 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.CdcrReplicationHandlerTest 60446B9388DD29F2-001\jetty-002\cores\collection1\data\index,segFN=segments_1,generation=1}
   [junit4]   2> 1513114 T8201 n:127.0.0.1:59799_ x:collection1 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 1513122 T8201 n:127.0.0.1:59799_ x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "cdcr-processor-chain"
   [junit4]   2> 1513122 T8201 n:127.0.0.1:59799_ x:collection1 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 1513127 T8201 n:127.0.0.1:59799_ x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 1513129 T8201 n:127.0.0.1:59799_ x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 1513131 T8201 n:127.0.0.1:59799_ x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 1513132 T8201 n:127.0.0.1:59799_ x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 1513134 T8201 n:127.0.0.1:59799_ x:collection1 oasc.PluginBag.createPlugin requestHandler : '/replication' created with startup=lazy 
   [junit4]   2> 1513142 T8201 n:127.0.0.1:59799_ x:collection1 oasc.SolrResourceLoader.findClass WARN Solr loaded a deprecated plugin/analysis class [org.apache.solr.handler.admin.AdminHandlers]. Please consult documentation how to replace it accordingly.
   [junit4]   2> 1513142 T8201 n:127.0.0.1:59799_ x:collection1 oasc.SolrResourceLoader.findClass WARN Solr loaded a deprecated plugin/analysis class [org.apache.solr.handler.admin.AdminHandlers]. Please consult documentation how to replace it accordingly.
   [junit4]   2> 1513143 T8201 n:127.0.0.1:59799_ x:collection1 oasc.RequestHandlers.initHandlersFromConfig Registered paths: /admin/mbeans,standard,/update/csv,/update/json/docs,/admin/luke,/admin/segments,/get,/admin/system,/replication,/admin/properties,/config,/schema,/admin/plugins,/admin/logging,/update/json,/admin,/admin/threads,/admin/ping,/cdcr,/update,/admin/file
   [junit4]   2> 1513143 T8201 n:127.0.0.1:59799_ x:collection1 oasc.SolrCore.initStatsCache Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 1513144 T8201 n:127.0.0.1:59799_ x:collection1 oasu.UpdateHandler.<init> Using UpdateLog implementation: org.apache.solr.update.CdcrUpdateLog
   [junit4]   2> 1513145 T8201 n:127.0.0.1:59799_ x:collection1 oasu.UpdateLog.init Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1513146 T8201 n:127.0.0.1:59799_ x:collection1 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 1513146 T8201 n:127.0.0.1:59799_ x:collection1 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 1513147 T8201 n:127.0.0.1:59799_ x:collection1 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=43, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
   [junit4]   2> 1513148 T8201 n:127.0.0.1:59799_ x:collection1 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.CdcrReplicationHandlerTest 60446B9388DD29F2-001\jetty-002\cores\collection1\data\index,segFN=segments_1,generation=1}
   [junit4]   2> 1513148 T8201 n:127.0.0.1:59799_ x:collection1 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 1513149 T8201 n:127.0.0.1:59799_ x:collection1 oass.SolrIndexSearcher.<init> Opening Searcher@1a3e8e4[collection1] main
   [junit4]   2> 1513149 T8201 n:127.0.0.1:59799_ x:collection1 oasu.UpdateLog.onFirstSearcher On first searcher opened, looking up max value of version field
   [junit4]   2> 1513149 T8201 n:127.0.0.1:59799_ x:collection1 oasu.VersionInfo.getMaxVersionFromIndex Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 1513149 T8201 n:127.0.0.1:59799_ x:collection1 oasu.VersionInfo.getMaxVersionFromIndex WARN No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 1513149 T8201 n:127.0.0.1:59799_ x:collection1 oasu.UpdateLog.seedBucketsWithHighestVersion WARN Could not find max version in index or recent updates, using new clock 1501960487262224384
   [junit4]   2> 1513151 T8201 n:127.0.0.1:59799_ x:collection1 oasu.UpdateLog.seedBucketsWithHighestVersion Took 2 ms to seed version buckets with highest version 1501960487262224384
   [junit4]   2> 1513152 T8201 n:127.0.0.1:59799_ x:collection1 oascc.ZkStateReader.readConfigName Load collection config from:/collections/tmp_collection
   [junit4]   2> 1513153 T8201 n:127.0.0.1:59799_ x:collection1 oascc.ZkStateReader.readConfigName path=/collections/tmp_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1513153 T8201 n:127.0.0.1:59799_ x:collection1 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 1513155 T8201 n:127.0.0.1:59799_ x:collection1 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 1513155 T8201 n:127.0.0.1:59799_ x:collection1 oasr.RestManager.init Initializing RestManager with initArgs: {}
   [junit4]   2> 1513155 T8201 n:127.0.0.1:59799_ x:collection1 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1513156 T8201 n:127.0.0.1:59799_ x:collection1 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 1513156 T8201 n:127.0.0.1:59799_ x:collection1 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1513156 T8201 n:127.0.0.1:59799_ x:collection1 oasr.RestManager.init Initializing 0 registered ManagedResources
   [junit4]   2> 1513161 T8201 n:127.0.0.1:59799_ x:collection1 oasha.AdminHandlers.inform WARN <requestHandler name="/admin/" 
   [junit4]   2> 	 class="solr.admin.AdminHandlers" /> is deprecated . It is not required anymore
   [junit4]   2> 1513162 T8202 n:127.0.0.1:59799_ x:collection1 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1a3e8e4[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1513162 T8201 n:127.0.0.1:59799_ x:collection1 oasc.ZkController.getConfDirListeners watch zkdir /configs/conf1
   [junit4]   2> 1513163 T8201 n:127.0.0.1:59799_ x:collection1 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 1513165 T8205 n:127.0.0.1:59799_ c:tmp_collection s:shard1 x:collection1 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:59799 collection:tmp_collection shard:shard1
   [junit4]   2> 1513165 T8146 n:127.0.0.1:59799_ oass.SolrDispatchFilter.init user.dir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0
   [junit4]   2> 1513166 T8146 n:127.0.0.1:59799_ oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 1513166 T8146 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 1513169 T8146 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1513170 T8208 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1abb290 name:ZooKeeperConnection Watcher:127.0.0.1:59783 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1513170 T8205 n:127.0.0.1:59799_ c:tmp_collection s:shard1 x:collection1 oasc.ZkController.register We are http://127.0.0.1:59799/collection1/ and leader is http://127.0.0.1:59790/collection1/
   [junit4]   2> 1513171 T8205 n:127.0.0.1:59799_ c:tmp_collection s:shard1 x:collection1 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:59799
   [junit4]   2> 1513172 T8205 n:127.0.0.1:59799_ c:tmp_collection s:shard1 x:collection1 oasc.ZkController.checkRecovery Core needs to recover:collection1
   [junit4]   2> 1513172 T8205 n:127.0.0.1:59799_ c:tmp_collection s:shard1 x:collection1 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
   [junit4]   2> 1513171 T8146 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1513173 T8146 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> ASYNC  NEW_CORE C2051 name=collection1 org.apache.solr.core.SolrCore@1bb6f71 url=http://127.0.0.1:59799/collection1 node=127.0.0.1:59799_ C2051_STATE=coll:tmp_collection core:collection1 props:{core=collection1, base_url=http://127.0.0.1:59799, node_name=127.0.0.1:59799_, state=down}
   [junit4]   2> 1513173 T8209 n:127.0.0.1:59799_ c:tmp_collection s:shard1 x:collection1 C2051 oasc.RecoveryStrategy.run Starting recovery process.  core=collection1 recoveringAfterStartup=true
   [junit4]   2> 1513175 T8209 n:127.0.0.1:59799_ c:tmp_collection s:shard1 x:collection1 C2051 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
   [junit4]   2> 1513175 T8209 n:127.0.0.1:59799_ c:tmp_collection s:shard1 x:collection1 C2051 oasc.RecoveryStrategy.doRecovery Publishing state of core collection1 as recovering, leader is http://127.0.0.1:59790/collection1/ and I am http://127.0.0.1:59799/collection1/
   [junit4]   2> 1513175 T8209 n:127.0.0.1:59799_ c:tmp_collection s:shard1 x:collection1 C2051 oasc.ZkController.publish publishing core=collection1 state=recovering collection=tmp_collection
   [junit4]   2> 1513175 T8209 n:127.0.0.1:59799_ c:tmp_collection s:shard1 x:collection1 C2051 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 1513178 T8174 n:127.0.0.1:59790_ oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 1513180 T8209 n:127.0.0.1:59799_ c:tmp_collection s:shard1 x:collection1 C2051 oasc.RecoveryStrategy.sendPrepRecoveryCmd Sending prep recovery command to http://127.0.0.1:59790; WaitForState: action=PREPRECOVERY&core=collection1&nodeName=127.0.0.1%3A59799_&coreNodeName=core_node2&state=recovering&checkLive=true&onlyIfLeader=true&onlyIfLeaderActive=true
   [junit4]   2> 1513181 T8175 n:127.0.0.1:59790_ oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "core_node_name":"core_node2",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"http://127.0.0.1:59799",
   [junit4]   2> 	  "node_name":"127.0.0.1:59799_",
   [junit4]   2> 	  "numShards":"1",
   [junit4]   2> 	  "state":"recovering",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "collection":"tmp_collection",
   [junit4]   2> 	  "operation":"state"} current state version: 4
   [junit4]   2> 1513182 T8175 n:127.0.0.1:59790_ oasco.ReplicaMutator.updateState Update state numShards=1 message={
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "core_node_name":"core_node2",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"http://127.0.0.1:59799",
   [junit4]   2> 	  "node_name":"127.0.0.1:59799_",
   [junit4]   2> 	  "numShards":"1",
   [junit4]   2> 	  "state":"recovering",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "collection":"tmp_collection",
   [junit4]   2> 	  "operation":"state"}
   [junit4]   2> 1513186 T8161 n:127.0.0.1:59790_ oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: core_node2, state: recovering, checkLive: true, onlyIfLeader: true, onlyIfLeaderActive: true
   [junit4]   2> 1513189 T8161 n:127.0.0.1:59790_ oasha.CoreAdminHandler.handleWaitForStateAction Will wait a max of 183 seconds to see collection1 (shard1 of tmp_collection) have state: recovering
   [junit4]   2> 1513189 T8161 n:127.0.0.1:59790_ oasha.CoreAdminHandler.handleWaitForStateAction In WaitForState(recovering): collection=tmp_collection, shard=shard1, thisCore=collection1, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=down, localState=active, nodeName=127.0.0.1:59799_, coreNodeName=core_node2, onlyIfActiveCheckResult=false, nodeProps: core_node2:{"core":"collection1","base_url":"http://127.0.0.1:59799","node_name":"127.0.0.1:59799_","state":"down"}
   [junit4]   1> / (2)
   [junit4]   1> DATA:
   [junit4]   1>     
   [junit4]   1>  /zookeeper (1)
   [junit4]   1>  DATA:
   [junit4]   1>      
   [junit4]   1>  /solr (7)
   [junit4]   1>   /solr/configs (1)
   [junit4]   1>    /solr/configs/conf1 (11)
   [junit4]   1>     /solr/configs/conf1/currency.xml (0)
   [junit4]   1>     DATA: ...supressed...
   [junit4]   1>     /solr/configs/conf1/enumsConfig.xml (0)
   [junit4]   1>     DATA: ...supressed...
   [junit4]   1>     /solr/configs/conf1/protwords.txt (0)
   [junit4]   1>     DATA: ...supressed...
   [junit4]   1>     /solr/configs/conf1/old_synonyms.txt (0)
   [junit4]   1>     DATA: ...supressed...
   [junit4]   1>     /solr/configs/conf1/solrconfig.xml (0)
   [junit4]   1>     DATA: ...supressed...
   [junit4]   1>     /solr/configs/conf1/synonyms.txt (0)
   [junit4]   1>     DATA: ...supressed...
   [junit4]   1>     /solr/configs/conf1/stopwords.txt (0)
   [junit4]   1>     DATA: ...supressed...
   [junit4]   1>     /solr/configs/conf1/schema.xml (0)
   [junit4]   1>     DATA: ...supressed...
   [junit4]   1>     /solr/configs/conf1/solrconfig.snippet.randomindexconfig.xml (0)
   [junit4]   1>     DATA: ...supressed...
   [junit4]   1>     /solr/configs/conf1/mapping-ISOLatin1Accent.txt (0)
   [junit4]   1>     DATA: ...supressed...
   [junit4]   1>     /solr/configs/conf1/open-exchange-rates.json (0)
   [junit4]   1>     DATA:
   [junit4]   1>         {
   [junit4]   1>           "disclaimer": "This data is not real, it was synthetically created to match currency.xml.  It is modeled after the data format available from openexchangerates.org.  See https://openexchangerates.org/documentation for details",
   [junit4]   1>           "license": "http://www.apache.org/licenses/LICENSE-2.0",
   [junit4]   1>           "timestamp": 1332070464,
   [junit4]   1>         
   [junit4]   1>         
   [junit4]   1>           "IMPORTANT NOTE": "In order for tests to work, this data must be kept in sync with ./currency.xml",
   [junit4]   1>         
   [junit4]   1>         
   [junit4]   1>           "base": "USD",
   [junit4]   1>           "rates": {
   [junit4]   1>             "USD": 1,
   [junit4]   1>             "JPY": 81.29,
   [junit4]   1>             "EUR": 2.5,
   [junit4]   1>             "GBP": 0.5,
   [junit4]   1>             "MXN": 2.0
   [junit4]   1>           }
   [junit4]   1>         }
   [junit4]   1>         
   [junit4]   1>   /solr/overseer (6)
   [junit4]   1>   DATA:
   [junit4]   1>       
   [junit4]   1>    /solr/overseer/collection-queue-work (0)
   [junit4]   1>    /solr/overseer/queue-work (1)
   [junit4]   1>     /solr/overseer/queue-work/qn-0000000005 (0)
   [junit4]   1>     DATA:
   [junit4]   1>         {
   [junit4]   1>           "core":"collection1",
   [junit4]   1>           "core_node_name":"core_node2",
   [junit4]   1>           "roles":null,
   [junit4]   1>           "base_url":"http://127.0.0.1:59799",
   [junit4]   1>           "node_name":"127.0.0.1:59799_",
   [junit4]   1>           "numShards":"1",
   [junit4]   1>           "state":"recovering",
   [junit4]   1>           "shard":"shard1",
   [junit4]   1>           "collection":"tmp_collection",
   [junit4]   1>           "operation":"state"}
   [junit4]   1>    /solr/overseer/collection-map-failure (0)
   [junit4]   1>    /solr/overseer/collection-map-completed (0)
   [junit4]   1>    /solr/overseer/queue (0)
   [junit4]   1>    /solr/overseer/collection-map-running (0)
   [junit4]   1>   /solr/aliases.json (0)
   [junit4]   1>   /solr/collections (3)
   [junit4]   1>    /solr/collections/collection1 (1)
   [junit4]   1>    DATA:
   [junit4]   1>        {"configName":"conf1"}
   [junit4]   1>     /solr/collections/collection1/shards (0)
   [junit4]   1>    /solr/collections/tmp_collection (3)
   [junit4]   1>    DATA:
   [junit4]   1>        {"configName":"conf1"}
   [junit4]   1>     /solr/collections/tmp_collection/leaders (1)
   [junit4]   1>      /solr/collections/tmp_collection/leaders/shard1 (0)
   [junit4]   1>      DATA:
   [junit4]   1>          {
   [junit4]   1>            "core":"collection1",
   [junit4]   1>            "base_url":"http://127.0.0.1:59790",
   [junit4]   1>            "node_name":"127.0.0.1:59790_"}
   [junit4]   1>     /solr/collections/tmp_collection/cdcr (1)
   [junit4]   1>      /solr/collections/tmp_collection/cdcr/state (2)
   [junit4]   1>       /solr/collections/tmp_collection/cdcr/state/process (0)
   [junit4]   1>       DATA:
   [junit4]   1>           stopped
   [junit4]   1>       /solr/collections/tmp_collection/cdcr/state/buffer (0)
   [junit4]   1>       DATA:
   [junit4]   1>           enabled
   [junit4]   1>     /solr/collections/tmp_collection/leader_elect (1)
   [junit4]   1>      /solr/collections/tmp_collection/leader_elect/shard1 (1)
   [junit4]   1>       /solr/collections/tmp_collection/leader_elect/shard1/election (2)
   [junit4]   1>        /solr/collections/tmp_collection/leader_elect/shard1/election/93872530158583813-core_node2-n_0000000001 (0)
   [junit4]   1>        /solr/collections/tmp_collection/leader_elect/shard1/election/93872530158583811-core_node1-n_0000000000 (0)
   [junit4]   1>    /solr/collections/control_collection (1)
   [junit4]   1>    DATA:
   [junit4]   1>        {"configName":"conf1"}
   [junit4]   1>     /solr/collections/control_collection/shards (0)
   [junit4]   1>   /solr/live_nodes (2)
   [junit4]   1>    /solr/live_nodes/127.0.0.1:59799_ (0)
   [junit4]   1>    /solr/live_nodes/127.0.0.1:59790_ (0)
   [junit4]   1>   /solr/overseer_elect (2)
   [junit4]   1>    /solr/overseer_elect/leader (0)
   [junit4]   1>    DATA:
   [junit4]   1>        {"id":"93872530158583811-127.0.0.1:59790_-n_0000000000"}
   [junit4]   1>    /solr/overseer_elect/election (2)
   [junit4]   1>     /solr/overseer_elect/election/93872530158583813-127.0.0.1:59799_-n_0000000001 (0)
   [junit4]   1>     /solr/overseer_elect/election/93872530158583811-127.0.0.1:59790_-n_0000000000 (0)
   [junit4]   1>   /solr/clusterstate.json (0)
   [junit4]   1>   DATA:
   [junit4]   1>       {"tmp_collection":{
   [junit4]   1>           "replicationFactor":"1",
   [junit4]   1>           "router":{"name":"compositeId"},
   [junit4]   1>           "maxShardsPerNode":"1",
   [junit4]   1>           "autoAddReplicas":"false",
   [junit4]   1>           "autoCreated":"true",
   [junit4]   1>           "shards":{"shard1":{
   [junit4]   1>               "range":"80000000-7fffffff",
   [junit4]   1>               "state":"active",
   [junit4]   1>               "replicas":{
   [junit4]   1>                 "core_node1":{
   [junit4]   1>                   "core":"collection1",
   [junit4]   1>                   "base_url":"http://127.0.0.1:59790",
   [junit4]   1>                   "node_name":"127.0.0.1:59790_",
   [junit4]   1>                   "state":"active",
   [junit4]   1>                   "leader":"true"},
   [junit4]   1>                 "core_node2":{
   [junit4]   1>                   "core":"collection1",
   [junit4]   1>                   "base_url":"http://127.0.0.1:59799",
   [junit4]   1>                   "node_name":"127.0.0.1:59799_",
   [junit4]   1>                   "state":"down"}}}}}}
   [junit4]   1> 
   [junit4]   2> 1513233 T8146 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 1513235 T8146 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1513237 T8214 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1f3b57a name:ZooKeeperConnection Watcher:127.0.0.1:59783/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1513238 T8146 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1513238 T8146 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 1513238 T8146 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 1513247 T8187 n:127.0.0.1:59799_ oasha.CollectionsHandler.handleRequestBody Invoked Collection Action :delete with paramsname=tmp_collection&action=DELETE&wt=javabin&version=2 
   [junit4]   2> 1513249 T8174 n:127.0.0.1:59790_ oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/collection-queue-work state SyncConnected
   [junit4]   2> 1513251 T8176 n:127.0.0.1:59790_ oasc.OverseerCollectionProcessor.run Overseer Collection Processor: Get the message id:/overseer/collection-queue-work/qn-0000000000 message:{
   [junit4]   2> 	  "name":"tmp_collection",
   [junit4]   2> 	  "operation":"delete"}
   [junit4]   2> 1513252 T8216 n:127.0.0.1:59790_ c:tmp_collection oasc.OverseerCollectionProcessor.processMessage WARN OverseerCollectionProcessor.processMessage : delete , {
   [junit4]   2> 	  "name":"tmp_collection",
   [junit4]   2> 	  "operation":"delete"}
   [junit4]   2> 1513255 T8216 n:127.0.0.1:59790_ c:tmp_collection oasc.OverseerCollectionProcessor.collectionCmd Executing Collection Cmd : action=UNLOAD&deleteInstanceDir=true&deleteDataDir=true
   [junit4]   2> 1513255 T8216 n:127.0.0.1:59790_ c:tmp_collection oasc.OverseerCollectionProcessor.sliceCmd Collection Admin sending CoreAdmin cmd to http://127.0.0.1:59790 params:action=UNLOAD&deleteInstanceDir=true&deleteDataDir=true&core=collection1&qt=%2Fadmin%2Fcores
   [junit4]   2> 1513256 T8216 n:127.0.0.1:59790_ c:tmp_collection oasc.OverseerCollectionProcessor.sliceCmd Collection Admin sending CoreAdmin cmd to http://127.0.0.1:59799 params:action=UNLOAD&deleteInstanceDir=true&deleteDataDir=true&core=collection1&qt=%2Fadmin%2Fcores
   [junit4]   2> 1513261 T8162 n:127.0.0.1:59790_ oasc.SolrCore.close [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@123da0a
   [junit4]   2> 1513261 T8162 n:127.0.0.1:59790_ oash.CdcrRequestHandler$1.preClose Solr core is being closed - shutting down CDCR handler @ tmp_collection:shard1
   [junit4]   2> 1513262 T8162 n:127.0.0.1:59790_ oasc.ZkController.unregisterConfListener removed listener for config directory [/configs/conf1]
   [junit4]   2> 1513262 T8162 n:127.0.0.1:59790_ oasc.ZkController.unregisterConfListener No more listeners for config directory [/configs/conf1]
   [junit4]   2> 1513262 T8162 n:127.0.0.1:59790_ oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0,transaction_logs_total_size=0,transaction_logs_total_number=0}
   [junit4]   2> 1513262 T8162 n:127.0.0.1:59790_ oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
   [junit4]   2> 1513263 T8162 n:127.0.0.1:59790_ oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
   [junit4]   2> 1513264 T8162 n:127.0.0.1:59790_ oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
   [junit4]   2> 1513263 T8188 n:127.0.0.1:59799_ oasc.RecoveryStrategy.close WARN Stopping recovery for core=collection1 coreNodeName=core_node2
   [junit4]   2> 1513265 T8209 n:127.0.0.1:59799_ c:tmp_collection s:shard1 x:collection1 C2051 oasc.SolrException.log ERROR Error while trying to recover. core=collection1:java.util.concurrent.ExecutionException: org.apache.solr.client.solrj.SolrServerException: IOException occured when talking to server at: http://127.0.0.1:59790
   [junit4]   2> 		at java.util.concurrent.FutureTask.report(FutureTask.java:122)
   [junit4]   2> 		at java.util.concurrent.FutureTask.get(FutureTask.java:192)
   [junit4]   2> 		at org.apache.solr.cloud.RecoveryStrategy.sendPrepRecoveryCmd(RecoveryStrategy.java:592)
   [junit4]   2> 		at org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:363)
   [junit4]   2> 		at org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:229)
   [junit4]   2> 	Caused by: org.apache.solr.client.solrj.SolrServerException: IOException occured when talking to server at: http://127.0.0.1:59790
   [junit4]   2> 		at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
   [junit4]   2> 		at org.apache.solr.client.solrj.impl.HttpSolrClient$1.call(HttpSolrClient.java:271)
   [junit4]   2> 		at org.apache.solr.client.solrj.impl.HttpSolrClient$1.call(HttpSolrClient.java:267)
   [junit4]   2> 		at java.util.concurrent.FutureTask.run(FutureTask.java:266)
   [junit4]   2> 		at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:148)
   [junit4]   2> 		at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
   [junit4]   2> 		at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]   2> 		at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 	Caused by: java.net.SocketException: Socket closed
   [junit4]   2> 		at java.net.SocketInputStream.socketRead0(Native Method)
   [junit4]   2> 		at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
   [junit4]   2> 		at java.net.SocketInputStream.read(SocketInputStream.java:170)
   [junit4]   2> 		at java.net.SocketInputStream.read(SocketInputStream.java:141)
   [junit4]   2> 		at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:160)
   [junit4]   2> 		at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:84)
   [junit4]   2> 		at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:273)
   [junit4]   2> 		at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
   [junit4]   2> 		at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
   [junit4]   2> 		at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
   [junit4]   2> 		at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283)
   [junit4]   2> 		at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251)
   [junit4]   2> 		at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197)
   [junit4]   2> 		at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)
   [junit4]   2> 		at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)
   [junit4]   2> 		at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685)
   [junit4]   2> 		at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487)
   [junit4]   2> 		at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
   [junit4]   2> 		at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
   [junit4]   2> 		at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
   [junit4]   2> 		at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
   [junit4]   2> 		at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:466)
   [junit4]   2> 		... 7 more
   [junit4]   2> 	
   [junit4]   2> 1513266 T8209 n:127.0.0.1:59799_ c:tmp_collection s:shard1 x:collection1 C2051 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
   [junit4]   2> 1513268 T8188 n:127.0.0.1:59799_ oasc.SolrCore.close [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@1bb6f71
   [junit4]   2> 1513268 T8188 n:127.0.0.1:59799_ oash.CdcrRequestHandler$1.preClose Solr core is being closed - shutting down CDCR handler @ tmp_collection:shard1
   [junit4]   2> 1513268 T8188 n:127.0.0.1:59799_ oasc.ZkController.unregisterConfListener removed listener for config directory [/configs/conf1]
   [junit4]   2> 1513269 T8188 n:127.0.0.1:59799_ oasc.ZkController.unregisterConfListener No more listeners for config directory [/configs/conf1]
   [junit4]   2> 1513269 T8162 n:127.0.0.1:59790_ oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
   [junit4]   2> 1513270 T8188 n:127.0.0.1:59799_ oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0,transaction_logs_total_size=0,transaction_logs_total_number=0}
   [junit4]   2> 1513270 T8188 n:127.0.0.1:59799_ oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
   [junit4]   2> 1513271 T8188 n:127.0.0.1:59799_ oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
   [junit4]   2> 1513271 T8188 n:127.0.0.1:59799_ oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
   [junit4]   2> 1513273 T8188 n:127.0.0.1:59799_ oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
   [junit4]   2> 1513286 T8162 n:127.0.0.1:59790_ oasc.CachingDirectoryFactory.close Closing StandardDirectoryFactory - 2 directories currently being tracked
   [junit4]   2> 1513287 T8162 n:127.0.0.1:59790_ oasc.CachingDirectoryFactory.closeCacheValue looking to close C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.CdcrReplicationHandlerTest 60446B9388DD29F2-001\jetty-001\cores\collection1\data [CachedDir<<refCount=0;path=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.CdcrReplicationHandlerTest 60446B9388DD29F2-001\jetty-001\cores\collection1\data;done=false>>]
   [junit4]   2> 1513287 T8162 n:127.0.0.1:59790_ oasc.CachingDirectoryFactory.close Closing directory: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.CdcrReplicationHandlerTest 60446B9388DD29F2-001\jetty-001\cores\collection1\data
   [junit4]   2> 1513287 T8200 n:127.0.0.1:59799_ oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1513288 T8162 n:127.0.0.1:59790_ oasc.CachingDirectoryFactory.closeCacheValue looking to close C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.CdcrReplicationHandlerTest 60446B9388DD29F2-001\jetty-001\cores\collection1\data\index [CachedDir<<refCount=0;path=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.CdcrReplicationHandlerTest 60446B9388DD29F2-001\jetty-001\cores\collection1\data\index;done=false>>]
   [junit4]   2> 1513289 T8162 n:127.0.0.1:59790_ oasc.CachingDirectoryFactory.close Closing directory: C:

[...truncated too long message...]

or@1c2ec05[Shutting down, pool size = 1, active threads = 1, queued tasks = 0, completed tasks = 42]
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2047)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:823)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1369)
   [junit4]   2> 	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.execute(ExecutorUtil.java:135)
   [junit4]   2> 	at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$3.process(SolrZkClient.java:266)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
   [junit4]   2> 
   [junit4]   2> 1899080 T8146 oejs.AbstractConnector.doStop Stopped ServerConnector@6ba46{HTTP/1.1}{127.0.0.1:59799}
   [junit4]   2> 1899080 T8146 oejsh.ContextHandler.doStop Stopped o.e.j.s.ServletContextHandler@11ea9d2{/,null,UNAVAILABLE}
   [junit4]   2> 1899083 T8146 c:source_collection s:shard1 x:source_collection_shard1_replica1 oas.SolrTestCaseJ4.tearDown ###Ending doTest
   [junit4]   2> 1899085 T8146 c:source_collection s:shard1 x:source_collection_shard1_replica1 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:59783 59783
   [junit4]   2> 1901822 T8147 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:59783 59783
   [junit4]   2> 1901823 T8147 oasc.ZkTestServer$ZKServerMain.runFromConfig WARN Watch limit violations: 
   [junit4]   2> 	Maximum concurrent create/delete watches above limit:
   [junit4]   2> 	
   [junit4]   2> 		37	/solr/aliases.json
   [junit4]   2> 		37	/solr/clusterstate.json
   [junit4]   2> 		8	/solr/configs/conf1
   [junit4]   2> 		5	/solr/collections/source_collection/state.json
   [junit4]   2> 		4	/solr/collections/source_collection/leaders/shard1
   [junit4]   2> 		2	/solr/collections/tmp_collection/leaders/shard1
   [junit4]   2> 	
   [junit4]   2> 	Maximum concurrent data watches above limit:
   [junit4]   2> 	
   [junit4]   2> 		6	/solr/collections/source_collection/cdcr/state/buffer
   [junit4]   2> 		6	/solr/collections/source_collection/cdcr/state/process
   [junit4]   2> 		2	/solr/overseer_elect/election/93872530158583811-127.0.0.1:59790_-n_0000000000
   [junit4]   2> 		2	/solr/collections/source_collection/leader_elect/shard1/election/93872530158583811-core_node1-n_0000000000
   [junit4]   2> 		2	/solr/collections/source_collection/leader_elect/shard1/election/93872530158583831-core_node2-n_0000000000
   [junit4]   2> 		2	/solr/collections/tmp_collection/cdcr/state/buffer
   [junit4]   2> 		2	/solr/collections/tmp_collection/cdcr/state/process
   [junit4]   2> 	
   [junit4]   2> 	Maximum concurrent children watches above limit:
   [junit4]   2> 	
   [junit4]   2> 		35	/solr/live_nodes
   [junit4]   2> 		2	/solr/overseer/queue
   [junit4]   2> 		2	/solr/overseer/collection-queue-work
   [junit4]   2> 	
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=CdcrReplicationHandlerTest -Dtests.method=doTest -Dtests.seed=60446B9388DD29F2 -Dtests.slow=true -Dtests.locale=es_ES -Dtests.timezone=America/North_Dakota/Center -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
   [junit4] FAILURE  393s J0 | CdcrReplicationHandlerTest.doTest <<<
   [junit4]    > Throwable #1: java.lang.AssertionError: There are still nodes recoverying - waited for 330 seconds
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([60446B9388DD29F2:C700D337E5663A4B]:0)
   [junit4]    > 	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:172)
   [junit4]    > 	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:133)
   [junit4]    > 	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:128)
   [junit4]    > 	at org.apache.solr.cloud.BaseCdcrDistributedZkTest.waitForRecoveriesToFinish(BaseCdcrDistributedZkTest.java:419)
   [junit4]    > 	at org.apache.solr.cloud.BaseCdcrDistributedZkTest.restartServer(BaseCdcrDistributedZkTest.java:460)
   [junit4]    > 	at org.apache.solr.cloud.CdcrReplicationHandlerTest.doTestPartialReplication(CdcrReplicationHandlerTest.java:109)
   [junit4]    > 	at org.apache.solr.cloud.CdcrReplicationHandlerTest.doTest(CdcrReplicationHandlerTest.java:51)
   [junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:960)
   [junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:935)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 1901848 T8146 c:source_collection s:shard1 x:source_collection_shard1_replica1 oas.SolrTestCaseJ4.deleteCore ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.CdcrReplicationHandlerTest 60446B9388DD29F2-001
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene50): {_version_=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), id=PostingsFormat(name=Memory doPackFST= true)}, docValues:{}, sim=RandomSimilarityProvider(queryNorm=true,coord=yes): {}, locale=es_ES, timezone=America/North_Dakota/Center
   [junit4]   2> NOTE: Windows 7 6.1 x86/Oracle Corporation 1.8.0_45 (32-bit)/cpus=3,threads=1,free=380450184,total=536870912
   [junit4]   2> NOTE: All tests run in this JVM: [TestDocBasedVersionConstraints, TestPostingsSolrHighlighter, SolrInfoMBeanTest, SolrTestCaseJ4Test, TestFreeTextSuggestions, TestChildDocTransformer, TestExactSharedStatsCache, CursorMarkTest, PreAnalyzedFieldTest, StressHdfsTest, InfoHandlerTest, RequiredFieldsTest, DeleteInactiveReplicaTest, TestCustomSort, RequestHandlersTest, TestFunctionQuery, TestReloadAndDeleteDocs, SimpleCollectionCreateDeleteTest, TestArbitraryIndexDir, TestAuthorizationFramework, TestQueryTypes, SyncSliceTest, DistributedFacetPivotSmallAdvancedTest, SolrPluginUtilsTest, TestOmitPositions, CoreAdminHandlerTest, ZkControllerTest, TestHighlightDedupGrouping, TestOverriddenPrefixQueryForCustomFieldType, MultiTermTest, TestIndexingPerformance, BufferStoreTest, TestFieldResource, TestSolrConfigHandlerConcurrent, SuggesterFSTTest, RuleEngineTest, CacheHeaderTest, TestLazyCores, ClusterStateTest, TestConfigOverlay, ChangedSchemaMergeTest, SynonymTokenizerTest, TestCloudManagedSchema, BaseCdcrDistributedZkTest, TestRecoveryHdfs, SuggesterTest, TestTrackingShardHandlerFactory, DirectUpdateHandlerTest, DocExpirationUpdateProcessorFactoryTest, TestStressRecovery, CdcrReplicationDistributedZkTest, ZkStateWriterTest, HdfsSyncSliceTest, QueryElevationComponentTest, TestWriterPerf, DistribDocExpirationUpdateProcessorTest, TestUpdate, TestPivotHelperCode, PathHierarchyTokenizerFactoryTest, UUIDFieldTest, TestDistributedSearch, DefaultValueUpdateProcessorTest, HdfsNNFailoverTest, ReplicationFactorTest, TestQueryUtils, DistributedQueryComponentCustomSortTest, TestLRUCache, TestFieldCollectionResource, HardAutoCommitTest, CdcrUpdateLogTest, MultiThreadedOCPTest, SliceStateTest, TestGroupingSearch, NumericFieldsTest, TestReplicationHandlerBackup, TestCSVLoader, TestLeaderElectionZkExpiry, TestExactStatsCache, TestExceedMaxTermLength, CoreMergeIndexesAdminHandlerTest, TestBM25SimilarityFactory, ChaosMonkeyNothingIsSafeTest, TestSchemaResource, SOLR749Test, OutOfBoxZkACLAndCredentialsProvidersTest, TestJmxMonitoredMap, SharedFSAutoReplicaFailoverUtilsTest, RemoteQueryErrorTest, ConnectionManagerTest, TestShardHandlerFactory, TestCollationFieldDocValues, IgnoreCommitOptimizeUpdateProcessorFactoryTest, ResponseLogComponentTest, FieldAnalysisRequestHandlerTest, TestSweetSpotSimilarityFactory, TestPartialUpdateDeduplication, TestFastLRUCache, TestFieldSortValues, DistributedQueryElevationComponentTest, TestSchemaNameResource, LeaderFailoverAfterPartitionTest, HighlighterMaxOffsetTest, SpatialHeatmapFacetsTest, IndexSchemaRuntimeFieldTest, UniqFieldsUpdateProcessorFactoryTest, TestAtomicUpdateErrorCases, TestExtendedDismaxParser, CdcrReplicationHandlerTest]
   [junit4] Completed [244/502] on J0 in 393.25s, 1 test, 1 failure <<< FAILURES!

[...truncated 798 lines...]
BUILD FAILED
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\build.xml:526: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\build.xml:474: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\build.xml:61: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\extra-targets.xml:39: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build.xml:229: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\common-build.xml:512: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\common-build.xml:1415: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\common-build.xml:973: There were test failures: 502 suites, 1986 tests, 1 failure, 66 ignored (34 assumptions)

Total time: 68 minutes 36 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-trunk-Windows (64bit/jdk1.8.0_45) - Build # 4843 - Still Failing!

Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Windows/4843/
Java: 64bit/jdk1.8.0_45 -XX:-UseCompressedOops -XX:+UseG1GC

5 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.CdcrRequestHandlerTest

Error Message:
Some resources were not closed, shutdown, or released.

Stack Trace:
java.lang.AssertionError: Some resources were not closed, shutdown, or released.
	at __randomizedtesting.SeedInfo.seed([46273C4DCB58A707]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.apache.solr.SolrTestCaseJ4.afterClass(SolrTestCaseJ4.java:234)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:497)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:799)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.CdcrRequestHandlerTest

Error Message:
5 threads leaked from SUITE scope at org.apache.solr.cloud.CdcrRequestHandlerTest:     1) Thread[id=563, name=zkCallback-101-thread-1, state=TIMED_WAITING, group=TGRP-CdcrRequestHandlerTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)    2) Thread[id=586, name=zkCallback-101-thread-2, state=TIMED_WAITING, group=TGRP-CdcrRequestHandlerTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)    3) Thread[id=561, name=zkCallback-72-thread-3-processing-{node_name=127.0.0.1:52489_jn%2Fbc}-SendThread(127.0.0.1:52482), state=TIMED_WAITING, group=TGRP-CdcrRequestHandlerTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:994)    4) Thread[id=587, name=zkCallback-101-thread-3, state=TIMED_WAITING, group=TGRP-CdcrRequestHandlerTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)    5) Thread[id=562, name=zkCallback-72-thread-3-processing-{node_name=127.0.0.1:52489_jn%2Fbc}-EventThread, state=WAITING, group=TGRP-CdcrRequestHandlerTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:494)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 5 threads leaked from SUITE scope at org.apache.solr.cloud.CdcrRequestHandlerTest: 
   1) Thread[id=563, name=zkCallback-101-thread-1, state=TIMED_WAITING, group=TGRP-CdcrRequestHandlerTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
   2) Thread[id=586, name=zkCallback-101-thread-2, state=TIMED_WAITING, group=TGRP-CdcrRequestHandlerTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
   3) Thread[id=561, name=zkCallback-72-thread-3-processing-{node_name=127.0.0.1:52489_jn%2Fbc}-SendThread(127.0.0.1:52482), state=TIMED_WAITING, group=TGRP-CdcrRequestHandlerTest]
        at java.lang.Thread.sleep(Native Method)
        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:994)
   4) Thread[id=587, name=zkCallback-101-thread-3, state=TIMED_WAITING, group=TGRP-CdcrRequestHandlerTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
   5) Thread[id=562, name=zkCallback-72-thread-3-processing-{node_name=127.0.0.1:52489_jn%2Fbc}-EventThread, state=WAITING, group=TGRP-CdcrRequestHandlerTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:494)
	at __randomizedtesting.SeedInfo.seed([46273C4DCB58A707]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.CdcrRequestHandlerTest

Error Message:
There are still zombie threads that couldn't be terminated:    1) Thread[id=563, name=zkCallback-101-thread-1, state=TIMED_WAITING, group=TGRP-CdcrRequestHandlerTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)    2) Thread[id=586, name=zkCallback-101-thread-2, state=TIMED_WAITING, group=TGRP-CdcrRequestHandlerTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)    3) Thread[id=561, name=zkCallback-72-thread-3-processing-{node_name=127.0.0.1:52489_jn%2Fbc}-SendThread(127.0.0.1:52482), state=TIMED_WAITING, group=TGRP-CdcrRequestHandlerTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:994)    4) Thread[id=587, name=zkCallback-101-thread-3, state=TIMED_WAITING, group=TGRP-CdcrRequestHandlerTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   1) Thread[id=563, name=zkCallback-101-thread-1, state=TIMED_WAITING, group=TGRP-CdcrRequestHandlerTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
   2) Thread[id=586, name=zkCallback-101-thread-2, state=TIMED_WAITING, group=TGRP-CdcrRequestHandlerTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
   3) Thread[id=561, name=zkCallback-72-thread-3-processing-{node_name=127.0.0.1:52489_jn%2Fbc}-SendThread(127.0.0.1:52482), state=TIMED_WAITING, group=TGRP-CdcrRequestHandlerTest]
        at java.lang.Thread.sleep(Native Method)
        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:994)
   4) Thread[id=587, name=zkCallback-101-thread-3, state=TIMED_WAITING, group=TGRP-CdcrRequestHandlerTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
	at __randomizedtesting.SeedInfo.seed([46273C4DCB58A707]:0)


FAILED:  org.apache.solr.cloud.CdcrReplicationHandlerTest.doTest

Error Message:
There are still nodes recoverying - waited for 330 seconds

Stack Trace:
java.lang.AssertionError: There are still nodes recoverying - waited for 330 seconds
	at __randomizedtesting.SeedInfo.seed([46273C4DCB58A707:E16384E9A6E3B4BE]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:172)
	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:133)
	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:128)
	at org.apache.solr.cloud.BaseCdcrDistributedZkTest.waitForRecoveriesToFinish(BaseCdcrDistributedZkTest.java:419)
	at org.apache.solr.cloud.BaseCdcrDistributedZkTest.restartServer(BaseCdcrDistributedZkTest.java:460)
	at org.apache.solr.cloud.CdcrReplicationHandlerTest.doTestPartialReplication(CdcrReplicationHandlerTest.java:109)
	at org.apache.solr.cloud.CdcrReplicationHandlerTest.doTest(CdcrReplicationHandlerTest.java:51)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:497)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:960)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:935)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.cloud.CdcrRequestHandlerTest.doTest

Error Message:
expected:<st[art]ed> but was:<st[opp]ed>

Stack Trace:
org.junit.ComparisonFailure: expected:<st[art]ed> but was:<st[opp]ed>
	at __randomizedtesting.SeedInfo.seed([46273C4DCB58A707:E16384E9A6E3B4BE]:0)
	at org.junit.Assert.assertEquals(Assert.java:125)
	at org.junit.Assert.assertEquals(Assert.java:147)
	at org.apache.solr.cloud.BaseCdcrDistributedZkTest.assertState(BaseCdcrDistributedZkTest.java:255)
	at org.apache.solr.cloud.CdcrRequestHandlerTest.doTestLifeCycleActions(CdcrRequestHandlerTest.java:55)
	at org.apache.solr.cloud.CdcrRequestHandlerTest.doTest(CdcrRequestHandlerTest.java:39)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:497)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:960)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:935)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 9263 lines...]
   [junit4] Suite: org.apache.solr.cloud.CdcrRequestHandlerTest
   [junit4]   2> Creating dataDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CdcrRequestHandlerTest 46273C4DCB58A707-001\init-core-data-001
   [junit4]   2> 87720 T392 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (false) and clientAuth (false)
   [junit4]   2> 87720 T392 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /jn/bc
   [junit4]   2> 87725 T392 oasc.ZkTestServer.run STARTING ZK TEST SERVER
   [junit4]   2> 87733 T393 oasc.ZkTestServer$2$1.setClientPort client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 87733 T393 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
   [junit4]   2> 87812 T392 oasc.ZkTestServer.run start zk server on port:52482
   [junit4]   2> 87892 T392 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\solrconfig-cdcr.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 87900 T392 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\schema15.xml to /configs/conf1/schema.xml
   [junit4]   2> 87904 T392 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 87906 T392 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 87913 T392 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 87915 T392 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 87917 T392 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 87921 T392 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 87924 T392 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 87926 T392 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 87930 T392 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 87935 T392 oas.SolrTestCaseJ4.setUp ###Starting doTest
   [junit4]   2> 88184 T392 oas.SolrTestCaseJ4.writeCoreProperties Writing core.properties file to C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CdcrRequestHandlerTest 46273C4DCB58A707-001\jetty-001\cores\collection1
   [junit4]   2> 88187 T392 oejs.Server.doStart jetty-9.2.10.v20150310
   [junit4]   2> 88193 T392 oejsh.ContextHandler.doStart Started o.e.j.s.ServletContextHandler@633d5f65{/jn/bc,null,AVAILABLE}
   [junit4]   2> 88196 T392 oejs.AbstractConnector.doStart Started ServerConnector@3d958919{HTTP/1.1}{127.0.0.1:52489}
   [junit4]   2> 88196 T392 oejs.Server.doStart Started @94646ms
   [junit4]   2> 88196 T392 oascse.JettySolrRunner$1.lifeCycleStarted Jetty properties: {shards=shard1, hostContext=/jn/bc, hostPort=52489, coreRootDirectory=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CdcrRequestHandlerTest 46273C4DCB58A707-001\jetty-001\cores}
   [junit4]   2> 88197 T392 oass.SolrDispatchFilter.init SolrDispatchFilter.init()sun.misc.Launcher$AppClassLoader@4e0e2f2a
   [junit4]   2> 88197 T392 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CdcrRequestHandlerTest 46273C4DCB58A707-001\jetty-001\'
   [junit4]   2> 88225 T392 oasc.SolrXmlConfig.fromFile Loading container configuration from C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CdcrRequestHandlerTest 46273C4DCB58A707-001\jetty-001\solr.xml
   [junit4]   2> 88231 T392 oasc.CorePropertiesLocator.<init> Config-defined core root directory: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CdcrRequestHandlerTest 46273C4DCB58A707-001\jetty-001\cores
   [junit4]   2> 88231 T392 oasc.CoreContainer.<init> New CoreContainer 973605993
   [junit4]   2> 88231 T392 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CdcrRequestHandlerTest 46273C4DCB58A707-001\jetty-001\]
   [junit4]   2> 88231 T392 oasc.CoreContainer.load loading shared library: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CdcrRequestHandlerTest 46273C4DCB58A707-001\jetty-001\lib
   [junit4]   2> 88232 T392 oasc.SolrResourceLoader.addToClassLoader WARN Can't find (or read) directory to add to classloader: lib (resolved as: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CdcrRequestHandlerTest 46273C4DCB58A707-001\jetty-001\lib).
   [junit4]   2> 88239 T392 oashc.HttpShardHandlerFactory.init created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 88241 T392 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 88242 T392 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 88242 T392 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 88242 T392 oasc.CoreContainer.load Node Name: 127.0.0.1
   [junit4]   2> 88243 T392 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:52482/solr
   [junit4]   2> 88243 T392 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 88284 T392 n:127.0.0.1:52489_jn%2Fbc oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:52489_jn%2Fbc
   [junit4]   2> 88293 T392 n:127.0.0.1:52489_jn%2Fbc oasc.Overseer.close Overseer (id=null) closing
   [junit4]   2> 88296 T392 n:127.0.0.1:52489_jn%2Fbc oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:52489_jn%2Fbc
   [junit4]   2> 88297 T392 n:127.0.0.1:52489_jn%2Fbc oasc.Overseer.start Overseer (id=93870968331894787-127.0.0.1:52489_jn%2Fbc-n_0000000000) starting
   [junit4]   2> 88308 T392 n:127.0.0.1:52489_jn%2Fbc oasc.OverseerAutoReplicaFailoverThread.<init> Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 88309 T422 n:127.0.0.1:52489_jn%2Fbc oasc.OverseerCollectionProcessor.run Process current queue of collection creations
   [junit4]   2> 88310 T421 n:127.0.0.1:52489_jn%2Fbc oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
   [junit4]   2> 88317 T392 n:127.0.0.1:52489_jn%2Fbc oasc.CoreContainer.initializeAuthenticationPlugin No authentication plugin used.
   [junit4]   2> 88318 T392 n:127.0.0.1:52489_jn%2Fbc oasc.CoreContainer.intializeAuthorizationPlugin Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 88319 T392 n:127.0.0.1:52489_jn%2Fbc oasc.CorePropertiesLocator.discover Looking for core definitions underneath C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CdcrRequestHandlerTest 46273C4DCB58A707-001\jetty-001\cores
   [junit4]   2> 88320 T392 n:127.0.0.1:52489_jn%2Fbc oasc.CoreDescriptor.<init> CORE DESCRIPTOR: {name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, loadOnStartup=true, instanceDir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CdcrRequestHandlerTest 46273C4DCB58A707-001\jetty-001\cores\collection1, collection=tmp_collection, absoluteInstDir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CdcrRequestHandlerTest 46273C4DCB58A707-001\jetty-001\cores\collection1\, coreNodeName=, dataDir=data\, shard=}
   [junit4]   2> 88320 T392 n:127.0.0.1:52489_jn%2Fbc oasc.CorePropertiesLocator.discoverUnder Found core collection1 in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CdcrRequestHandlerTest 46273C4DCB58A707-001\jetty-001\cores\collection1\
   [junit4]   2> 88321 T392 n:127.0.0.1:52489_jn%2Fbc oasc.CorePropertiesLocator.discover Found 1 core definitions
   [junit4]   2> 88325 T424 n:127.0.0.1:52489_jn%2Fbc c:tmp_collection x:collection1 oasc.ZkController.publish publishing core=collection1 state=down collection=tmp_collection
   [junit4]   2> 88325 T424 n:127.0.0.1:52489_jn%2Fbc c:tmp_collection x:collection1 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 88326 T424 n:127.0.0.1:52489_jn%2Fbc oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 88326 T420 n:127.0.0.1:52489_jn%2Fbc oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 88328 T421 n:127.0.0.1:52489_jn%2Fbc oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"http://127.0.0.1:52489/jn/bc",
   [junit4]   2> 	  "node_name":"127.0.0.1:52489_jn%2Fbc",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "collection":"tmp_collection",
   [junit4]   2> 	  "operation":"state"} current state version: 0
   [junit4]   2> 88328 T421 n:127.0.0.1:52489_jn%2Fbc oasco.ReplicaMutator.updateState Update state numShards=2 message={
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"http://127.0.0.1:52489/jn/bc",
   [junit4]   2> 	  "node_name":"127.0.0.1:52489_jn%2Fbc",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "collection":"tmp_collection",
   [junit4]   2> 	  "operation":"state"}
   [junit4]   2> 88328 T421 n:127.0.0.1:52489_jn%2Fbc oasco.ClusterStateMutator.createCollection building a new cName: tmp_collection
   [junit4]   2> 88328 T421 n:127.0.0.1:52489_jn%2Fbc oasco.ReplicaMutator.updateState Assigning new node to shard shard=shard2
   [junit4]   2> 89182 T424 n:127.0.0.1:52489_jn%2Fbc oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
   [junit4]   2> 89182 T424 n:127.0.0.1:52489_jn%2Fbc oasc.ZkController.createCollectionZkNode Check for collection zkNode:tmp_collection
   [junit4]   2> 89182 T424 n:127.0.0.1:52489_jn%2Fbc oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:tmp_collection
   [junit4]   2> 89183 T424 n:127.0.0.1:52489_jn%2Fbc oasc.ZkController.getConfName Looking for collection configName
   [junit4]   2> 89183 T424 n:127.0.0.1:52489_jn%2Fbc oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
   [junit4]   2> 89186 T424 n:127.0.0.1:52489_jn%2Fbc oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CdcrRequestHandlerTest 46273C4DCB58A707-001\jetty-001\cores\collection1\'
   [junit4]   2> 89202 T424 n:127.0.0.1:52489_jn%2Fbc oasc.Config.<init> loaded config solrconfig.xml with version 0 
   [junit4]   2> 89208 T424 n:127.0.0.1:52489_jn%2Fbc oasc.SolrConfig.refreshRequestParams current version of requestparams : -1
   [junit4]   2> 89212 T424 n:127.0.0.1:52489_jn%2Fbc oasc.SolrConfig.<init> Using Lucene MatchVersion: 6.0.0
   [junit4]   2> 89218 T424 n:127.0.0.1:52489_jn%2Fbc oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 89219 T424 n:127.0.0.1:52489_jn%2Fbc oass.IndexSchema.readSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 89224 T424 n:127.0.0.1:52489_jn%2Fbc oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 89291 T424 n:127.0.0.1:52489_jn%2Fbc oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 89292 T424 n:127.0.0.1:52489_jn%2Fbc oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 89293 T424 n:127.0.0.1:52489_jn%2Fbc oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 89295 T424 n:127.0.0.1:52489_jn%2Fbc oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 89315 T424 n:127.0.0.1:52489_jn%2Fbc oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection tmp_collection
   [junit4]   2> 89315 T424 n:127.0.0.1:52489_jn%2Fbc x:collection1 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 89315 T424 n:127.0.0.1:52489_jn%2Fbc x:collection1 oasc.SolrCore.<init> [[collection1] ] Opening new SolrCore at [C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CdcrRequestHandlerTest 46273C4DCB58A707-001\jetty-001\cores\collection1\], dataDir=[null]
   [junit4]   2> 89315 T424 n:127.0.0.1:52489_jn%2Fbc x:collection1 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@359559e0
   [junit4]   2> 89316 T424 n:127.0.0.1:52489_jn%2Fbc x:collection1 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CdcrRequestHandlerTest 46273C4DCB58A707-001\jetty-001\cores\collection1\data\
   [junit4]   2> 89316 T424 n:127.0.0.1:52489_jn%2Fbc x:collection1 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CdcrRequestHandlerTest 46273C4DCB58A707-001\jetty-001\cores\collection1\data\index/
   [junit4]   2> 89316 T424 n:127.0.0.1:52489_jn%2Fbc x:collection1 oasc.SolrCore.initIndex WARN [collection1] Solr index directory 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CdcrRequestHandlerTest 46273C4DCB58A707-001\jetty-001\cores\collection1\data\index' doesn't exist. Creating new index...
   [junit4]   2> 89316 T424 n:127.0.0.1:52489_jn%2Fbc x:collection1 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CdcrRequestHandlerTest 46273C4DCB58A707-001\jetty-001\cores\collection1\data\index
   [junit4]   2> 89316 T424 n:127.0.0.1:52489_jn%2Fbc x:collection1 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=25, maxMergeAtOnceExplicit=39, maxMergedSegmentMB=2.896484375, floorSegmentMB=2.048828125, forceMergeDeletesPctAllowed=16.24685956030096, segmentsPerTier=49.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.8190586478696402
   [junit4]   2> 89317 T424 n:127.0.0.1:52489_jn%2Fbc x:collection1 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@551145c2 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@29a8b9cc),segFN=segments_1,generation=1}
   [junit4]   2> 89317 T424 n:127.0.0.1:52489_jn%2Fbc x:collection1 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 89321 T424 n:127.0.0.1:52489_jn%2Fbc x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "cdcr-processor-chain"
   [junit4]   2> 89352 T424 n:127.0.0.1:52489_jn%2Fbc x:collection1 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 89356 T424 n:127.0.0.1:52489_jn%2Fbc x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 89357 T424 n:127.0.0.1:52489_jn%2Fbc x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 89358 T424 n:127.0.0.1:52489_jn%2Fbc x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 89360 T424 n:127.0.0.1:52489_jn%2Fbc x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 89361 T424 n:127.0.0.1:52489_jn%2Fbc x:collection1 oasc.PluginBag.createPlugin requestHandler : '/replication' created with startup=lazy 
   [junit4]   2> 89392 T424 n:127.0.0.1:52489_jn%2Fbc x:collection1 oasc.SolrResourceLoader.findClass WARN Solr loaded a deprecated plugin/analysis class [org.apache.solr.handler.admin.AdminHandlers]. Please consult documentation how to replace it accordingly.
   [junit4]   2> 89392 T424 n:127.0.0.1:52489_jn%2Fbc x:collection1 oasc.SolrResourceLoader.findClass WARN Solr loaded a deprecated plugin/analysis class [org.apache.solr.handler.admin.AdminHandlers]. Please consult documentation how to replace it accordingly.
   [junit4]   2> 89393 T424 n:127.0.0.1:52489_jn%2Fbc x:collection1 oasc.RequestHandlers.initHandlersFromConfig Registered paths: /admin/mbeans,standard,/update/csv,/update/json/docs,/admin/luke,/admin/segments,/get,/admin/system,/replication,/admin/properties,/config,/schema,/admin/plugins,/admin/logging,/update/json,/admin,/admin/threads,/admin/ping,/cdcr,/update,/admin/file
   [junit4]   2> 89393 T424 n:127.0.0.1:52489_jn%2Fbc x:collection1 oasc.SolrCore.initStatsCache Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 89414 T424 n:127.0.0.1:52489_jn%2Fbc x:collection1 oasu.UpdateHandler.<init> Using UpdateLog implementation: org.apache.solr.update.CdcrUpdateLog
   [junit4]   2> 89414 T424 n:127.0.0.1:52489_jn%2Fbc x:collection1 oasu.UpdateLog.init Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 89416 T424 n:127.0.0.1:52489_jn%2Fbc x:collection1 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 89416 T424 n:127.0.0.1:52489_jn%2Fbc x:collection1 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 89419 T424 n:127.0.0.1:52489_jn%2Fbc x:collection1 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=37, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
   [junit4]   2> 89420 T424 n:127.0.0.1:52489_jn%2Fbc x:collection1 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@551145c2 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@29a8b9cc),segFN=segments_1,generation=1}
   [junit4]   2> 89420 T424 n:127.0.0.1:52489_jn%2Fbc x:collection1 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 89421 T424 n:127.0.0.1:52489_jn%2Fbc x:collection1 oass.SolrIndexSearcher.<init> Opening Searcher@6eaf94e2[collection1] main
   [junit4]   2> 89421 T424 n:127.0.0.1:52489_jn%2Fbc x:collection1 oasu.UpdateLog.onFirstSearcher On first searcher opened, looking up max value of version field
   [junit4]   2> 89421 T424 n:127.0.0.1:52489_jn%2Fbc x:collection1 oasu.VersionInfo.getMaxVersionFromIndex Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 89421 T424 n:127.0.0.1:52489_jn%2Fbc x:collection1 oasu.VersionInfo.getMaxVersionFromIndex WARN No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 89421 T424 n:127.0.0.1:52489_jn%2Fbc x:collection1 oasu.UpdateLog.seedBucketsWithHighestVersion WARN Could not find max version in index or recent updates, using new clock 1501935495076118528
   [junit4]   2> 89422 T424 n:127.0.0.1:52489_jn%2Fbc x:collection1 oasu.UpdateLog.seedBucketsWithHighestVersion Took 1 ms to seed version buckets with highest version 1501935495076118528
   [junit4]   2> 89423 T424 n:127.0.0.1:52489_jn%2Fbc x:collection1 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 89423 T424 n:127.0.0.1:52489_jn%2Fbc x:collection1 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 89424 T424 n:127.0.0.1:52489_jn%2Fbc x:collection1 oasr.RestManager.init Initializing RestManager with initArgs: {}
   [junit4]   2> 89424 T424 n:127.0.0.1:52489_jn%2Fbc x:collection1 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 89435 T424 n:127.0.0.1:52489_jn%2Fbc x:collection1 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 89436 T424 n:127.0.0.1:52489_jn%2Fbc x:collection1 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 89436 T424 n:127.0.0.1:52489_jn%2Fbc x:collection1 oasr.RestManager.init Initializing 0 registered ManagedResources
   [junit4]   2> 89446 T424 n:127.0.0.1:52489_jn%2Fbc x:collection1 oash.CdcrBufferStateManager.createStateNode Created znode /collections/tmp_collection/cdcr/state/buffer
   [junit4]   2> 89453 T424 n:127.0.0.1:52489_jn%2Fbc x:collection1 oash.CdcrProcessStateManager.createStateNode Created znode /collections/tmp_collection/cdcr/state/process
   [junit4]   2> 89493 T424 n:127.0.0.1:52489_jn%2Fbc x:collection1 oasha.AdminHandlers.inform WARN <requestHandler name="/admin/" 
   [junit4]   2> 	 class="solr.admin.AdminHandlers" /> is deprecated . It is not required anymore
   [junit4]   2> 89493 T425 n:127.0.0.1:52489_jn%2Fbc x:collection1 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6eaf94e2[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 89493 T424 n:127.0.0.1:52489_jn%2Fbc x:collection1 oasc.ZkController.getConfDirListeners watch zkdir /configs/conf1
   [junit4]   2> 89493 T424 n:127.0.0.1:52489_jn%2Fbc x:collection1 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 89495 T429 n:127.0.0.1:52489_jn%2Fbc c:tmp_collection s:shard2 x:collection1 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:52489/jn/bc collection:tmp_collection shard:shard2
   [junit4]   2> 89496 T392 n:127.0.0.1:52489_jn%2Fbc oass.SolrDispatchFilter.init user.dir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J1
   [junit4]   2> 89496 T392 n:127.0.0.1:52489_jn%2Fbc oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 89505 T429 n:127.0.0.1:52489_jn%2Fbc c:tmp_collection s:shard2 x:collection1 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard2
   [junit4]   2> 89509 T420 n:127.0.0.1:52489_jn%2Fbc oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 89509 T429 n:127.0.0.1:52489_jn%2Fbc c:tmp_collection s:shard2 x:collection1 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
   [junit4]   2> 89509 T429 n:127.0.0.1:52489_jn%2Fbc c:tmp_collection s:shard2 x:collection1 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
   [junit4]   2> ASYNC  NEW_CORE C82 name=collection1 org.apache.solr.core.SolrCore@3823c5f0 url=http://127.0.0.1:52489/jn/bc/collection1 node=127.0.0.1:52489_jn%2Fbc C82_STATE=coll:tmp_collection core:collection1 props:{core=collection1, base_url=http://127.0.0.1:52489/jn/bc, node_name=127.0.0.1:52489_jn%2Fbc, state=down}
   [junit4]   2> 89509 T429 n:127.0.0.1:52489_jn%2Fbc c:tmp_collection s:shard2 x:collection1 C82 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:52489/jn/bc/collection1/
   [junit4]   2> 89509 T429 n:127.0.0.1:52489_jn%2Fbc c:tmp_collection s:shard2 x:collection1 C82 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
   [junit4]   2> 89509 T421 n:127.0.0.1:52489_jn%2Fbc oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "operation":"leader",
   [junit4]   2> 	  "shard":"shard2",
   [junit4]   2> 	  "collection":"tmp_collection"} current state version: 1
   [junit4]   2> 89509 T429 n:127.0.0.1:52489_jn%2Fbc c:tmp_collection s:shard2 x:collection1 C82 oasc.SyncStrategy.syncToMe http://127.0.0.1:52489/jn/bc/collection1/ has no replicas
   [junit4]   2> 89509 T429 n:127.0.0.1:52489_jn%2Fbc c:tmp_collection s:shard2 x:collection1 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:52489/jn/bc/collection1/ shard2
   [junit4]   2> 89514 T420 n:127.0.0.1:52489_jn%2Fbc oash.CdcrLeaderStateManager$LeaderStateWatcher.process Received new leader state @ tmp_collection:shard2
   [junit4]   2> 89520 T420 n:127.0.0.1:52489_jn%2Fbc oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 89522 T421 n:127.0.0.1:52489_jn%2Fbc oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "operation":"leader",
   [junit4]   2> 	  "shard":"shard2",
   [junit4]   2> 	  "collection":"tmp_collection",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:52489/jn/bc",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "state":"active"} current state version: 1
   [junit4]   2> 89616 T429 n:127.0.0.1:52489_jn%2Fbc c:tmp_collection s:shard2 x:collection1 oasc.ZkController.register We are http://127.0.0.1:52489/jn/bc/collection1/ and leader is http://127.0.0.1:52489/jn/bc/collection1/
   [junit4]   2> 89616 T429 n:127.0.0.1:52489_jn%2Fbc c:tmp_collection s:shard2 x:collection1 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:52489/jn/bc
   [junit4]   2> 89616 T429 n:127.0.0.1:52489_jn%2Fbc c:tmp_collection s:shard2 x:collection1 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
   [junit4]   2> 89616 T429 n:127.0.0.1:52489_jn%2Fbc c:tmp_collection s:shard2 x:collection1 oasc.ZkController.publish publishing core=collection1 state=active collection=tmp_collection
   [junit4]   2> 89616 T429 n:127.0.0.1:52489_jn%2Fbc c:tmp_collection s:shard2 x:collection1 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 89618 T420 n:127.0.0.1:52489_jn%2Fbc oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 89619 T421 n:127.0.0.1:52489_jn%2Fbc oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "core_node_name":"core_node1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"http://127.0.0.1:52489/jn/bc",
   [junit4]   2> 	  "node_name":"127.0.0.1:52489_jn%2Fbc",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "state":"active",
   [junit4]   2> 	  "shard":"shard2",
   [junit4]   2> 	  "collection":"tmp_collection",
   [junit4]   2> 	  "operation":"state"} current state version: 2
   [junit4]   2> 89620 T421 n:127.0.0.1:52489_jn%2Fbc oasco.ReplicaMutator.updateState Update state numShards=2 message={
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "core_node_name":"core_node1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"http://127.0.0.1:52489/jn/bc",
   [junit4]   2> 	  "node_name":"127.0.0.1:52489_jn%2Fbc",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "state":"active",
   [junit4]   2> 	  "shard":"shard2",
   [junit4]   2> 	  "collection":"tmp_collection",
   [junit4]   2> 	  "operation":"state"}
   [junit4]   2> 89761 T392 oas.SolrTestCaseJ4.writeCoreProperties Writing core.properties file to C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CdcrRequestHandlerTest 46273C4DCB58A707-001\jetty-002\cores\collection1
   [junit4]   2> 89763 T392 oejs.Server.doStart jetty-9.2.10.v20150310
   [junit4]   2> 89767 T392 oejsh.ContextHandler.doStart Started o.e.j.s.ServletContextHandler@c05f46d{/jn/bc,null,AVAILABLE}
   [junit4]   2> 89768 T392 oejs.AbstractConnector.doStart Started ServerConnector@124a1333{HTTP/1.1}{127.0.0.1:52498}
   [junit4]   2> 89768 T392 oejs.Server.doStart Started @96484ms
   [junit4]   2> 89768 T392 oascse.JettySolrRunner$1.lifeCycleStarted Jetty properties: {shards=shard2, hostContext=/jn/bc, hostPort=52498, coreRootDirectory=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CdcrRequestHandlerTest 46273C4DCB58A707-001\jetty-002\cores}
   [junit4]   2> 89768 T392 oass.SolrDispatchFilter.init SolrDispatchFilter.init()sun.misc.Launcher$AppClassLoader@4e0e2f2a
   [junit4]   2> 89768 T392 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CdcrRequestHandlerTest 46273C4DCB58A707-001\jetty-002\'
   [junit4]   2> 89789 T392 oasc.SolrXmlConfig.fromFile Loading container configuration from C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CdcrRequestHandlerTest 46273C4DCB58A707-001\jetty-002\solr.xml
   [junit4]   2> 89796 T392 oasc.CorePropertiesLocator.<init> Config-defined core root directory: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CdcrRequestHandlerTest 46273C4DCB58A707-001\jetty-002\cores
   [junit4]   2> 89796 T392 oasc.CoreContainer.<init> New CoreContainer 1469202968
   [junit4]   2> 89796 T392 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CdcrRequestHandlerTest 46273C4DCB58A707-001\jetty-002\]
   [junit4]   2> 89797 T392 oasc.CoreContainer.load loading shared library: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CdcrRequestHandlerTest 46273C4DCB58A707-001\jetty-002\lib
   [junit4]   2> 89797 T392 oasc.SolrResourceLoader.addToClassLoader WARN Can't find (or read) directory to add to classloader: lib (resolved as: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CdcrRequestHandlerTest 46273C4DCB58A707-001\jetty-002\lib).
   [junit4]   2> 89804 T392 oashc.HttpShardHandlerFactory.init created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 89807 T392 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 89808 T392 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 89808 T392 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 89808 T392 oasc.CoreContainer.load Node Name: 127.0.0.1
   [junit4]   2> 89808 T392 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:52482/solr
   [junit4]   2> 89808 T392 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 90761 T392 n:127.0.0.1:52498_jn%2Fbc oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:52498_jn%2Fbc
   [junit4]   2> 90766 T392 n:127.0.0.1:52498_jn%2Fbc oasc.Overseer.close Overseer (id=null) closing
   [junit4]   2> 90769 T392 n:127.0.0.1:52498_jn%2Fbc oasc.CoreContainer.initializeAuthenticationPlugin No authentication plugin used.
   [junit4]   2> 90770 T392 n:127.0.0.1:52498_jn%2Fbc oasc.CoreContainer.intializeAuthorizationPlugin Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 90771 T392 n:127.0.0.1:52498_jn%2Fbc oasc.CorePropertiesLocator.discover Looking for core definitions underneath C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CdcrRequestHandlerTest 46273C4DCB58A707-001\jetty-002\cores
   [junit4]   2> 90772 T392 n:127.0.0.1:52498_jn%2Fbc oasc.CoreDescriptor.<init> CORE DESCRIPTOR: {name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, loadOnStartup=true, instanceDir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CdcrRequestHandlerTest 46273C4DCB58A707-001\jetty-002\cores\collection1, collection=tmp_collection, absoluteInstDir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CdcrRequestHandlerTest 46273C4DCB58A707-001\jetty-002\cores\collection1\, coreNodeName=, dataDir=data\, shard=}
   [junit4]   2> 90773 T392 n:127.0.0.1:52498_jn%2Fbc oasc.CorePropertiesLocator.discoverUnder Found core collection1 in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CdcrRequestHandlerTest 46273C4DCB58A707-001\jetty-002\cores\collection1\
   [junit4]   2> 90773 T392 n:127.0.0.1:52498_jn%2Fbc oasc.CorePropertiesLocator.discover Found 1 core definitions
   [junit4]   2> 90776 T447 n:127.0.0.1:52498_jn%2Fbc c:tmp_collection x:collection1 oasc.ZkController.publish publishing core=collection1 state=down collection=tmp_collection
   [junit4]   2> 90776 T447 n:127.0.0.1:52498_jn%2Fbc c:tmp_collection x:collection1 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 90777 T447 n:127.0.0.1:52498_jn%2Fbc oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 90777 T420 n:127.0.0.1:52489_jn%2Fbc oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 90779 T421 n:127.0.0.1:52489_jn%2Fbc oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"http://127.0.0.1:52498/jn/bc",
   [junit4]   2> 	  "node_name":"127.0.0.1:52498_jn%2Fbc",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "collection":"tmp_collection",
   [junit4]   2> 	  "operation":"state"} current state version: 3
   [junit4]   2> 90779 T421 n:127.0.0.1:52489_jn%2Fbc oasco.ReplicaMutator.updateState Update state numShards=2 message={
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"http://127.0.0.1:52498/jn/bc",
   [junit4]   2> 	  "node_name":"127.0.0.1:52498_jn%2Fbc",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "collection":"tmp_collection",
   [junit4]   2> 	  "operation":"state"}
   [junit4]   2> 90779 T421 n:127.0.0.1:52489_jn%2Fbc oasco.ReplicaMutator.updateState Collection already exists with numShards=2
   [junit4]   2> 90779 T421 n:127.0.0.1:52489_jn%2Fbc oasco.ReplicaMutator.updateState Assigning new node to shard shard=shard1
   [junit4]   2> 91718 T447 n:127.0.0.1:52498_jn%2Fbc oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
   [junit4]   2> 91718 T447 n:127.0.0.1:52498_jn%2Fbc oasc.ZkController.createCollectionZkNode Check for collection zkNode:tmp_collection
   [junit4]   2> 91718 T447 n:127.0.0.1:52498_jn%2Fbc oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 91720 T447 n:127.0.0.1:52498_jn%2Fbc oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CdcrRequestHandlerTest 46273C4DCB58A707-001\jetty-002\cores\collection1\'
   [junit4]   2> 91735 T447 n:127.0.0.1:52498_jn%2Fbc oasc.Config.<init> loaded config solrconfig.xml with version 0 
   [junit4]   2> 91742 T447 n:127.0.0.1:52498_jn%2Fbc oasc.SolrConfig.refreshRequestParams current version of requestparams : -1
   [junit4]   2> 91744 T447 n:127.0.0.1:52498_jn%2Fbc oasc.SolrConfig.<init> Using Lucene MatchVersion: 6.0.0
   [junit4]   2> 91751 T447 n:127.0.0.1:52498_jn%2Fbc oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 91752 T447 n:127.0.0.1:52498_jn%2Fbc oass.IndexSchema.readSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 91757 T447 n:127.0.0.1:52498_jn%2Fbc oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 91846 T447 n:127.0.0.1:52498_jn%2Fbc oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 91847 T447 n:127.0.0.1:52498_jn%2Fbc oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 91848 T447 n:127.0.0.1:52498_jn%2Fbc oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 91869 T447 n:127.0.0.1:52498_jn%2Fbc oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 91904 T447 n:127.0.0.1:52498_jn%2Fbc oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection tmp_collection
   [junit4]   2> 91905 T447 n:127.0.0.1:52498_jn%2Fbc x:collection1 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 91905 T447 n:127.0.0.1:52498_jn%2Fbc x:collection1 oasc.SolrCore.<init> [[collection1] ] Opening new SolrCore at [C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CdcrRequestHandlerTest 46273C4DCB58A707-001\jetty-002\cores\collection1\], dataDir=[null]
   [junit4]   2> 91905 T447 n:127.0.0.1:52498_jn%2Fbc x:collection1 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@359559e0
   [junit4]   2> 91906 T447 n:127.0.0.1:52498_jn%2Fbc x:collection1 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CdcrRequestHandlerTest 46273C4DCB58A707-001\jetty-002\cores\collection1\data\
   [junit4]   2> 91906 T447 n:127.0.0.1:52498_jn%2Fbc x:collection1 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CdcrRequestHandlerTest 46273C4DCB58A707-001\jetty-002\cores\collection1\data\index/
   [junit4]   2> 91906 T447 n:127.0.0.1:52498_jn%2Fbc x:collection1 oasc.SolrCore.initIndex WARN [collection1] Solr index directory 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CdcrRequestHandlerTest 46273C4DCB58A707-001\jetty-002\cores\collection1\data\index' doesn't exist. Creating new index...
   [junit4]   2> 91907 T447 n:127.0.0.1:52498_jn%2Fbc x:collection1 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CdcrRequestHandlerTest 46273C4DCB58A707-001\jetty-002\cores\collection1\data\index
   [junit4]   2> 91907 T447 n:127.0.0.1:52498_jn%2Fbc x:collection1 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=25, maxMergeAtOnceExplicit=39, maxMergedSegmentMB=2.896484375, floorSegmentMB=2.048828125, forceMergeDeletesPctAllowed=16.24685956030096, segmentsPerTier=49.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.8190586478696402
   [junit4]   2> 91907 T447 n:127.0.0.1:52498_jn%2Fbc x:collection1 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@2f6ee954 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@4109d5b3),segFN=segments_1,generation=1}
   [junit4]   2> 91908 T447 n:127.0.0.1:52498_jn%2Fbc x:collection1 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 91914 T447 n:127.0.0.1:52498_jn%2Fbc x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "cdcr-processor-chain"
   [junit4]   2> 91915 T447 n:127.0.0.1:52498_jn%2Fbc x:collection1 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 91919 T447 n:127.0.0.1:52498_jn%2Fbc x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 91920 T447 n:127.0.0.1:52498_jn%2Fbc x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 91921 T447 n:127.0.0.1:52498_jn%2Fbc x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 91922 T447 n:127.0.0.1:52498_jn%2Fbc x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 91923 T447 n:127.0.0.1:52498_jn%2Fbc x:collection1 oasc.PluginBag.createPlugin requestHandler : '/replication' created with startup=lazy 
   [junit4]   2> 91930 T447 n:127.0.0.1:52498_jn%2Fbc x:collection1 oasc.SolrResourceLoader.findClass WARN Solr loaded a deprecated plugin/analysis class [org.apache.solr.handler.admin.AdminHandlers]. Please consult documentation how to replace it accordingly.
   [junit4]   2> 91931 T447 n:127.0.0.1:52498_jn%2Fbc x:collection1 oasc.SolrResourceLoader.findClass WARN Solr loaded a deprecated plugin/analysis class [org.apache.solr.handler.admin.AdminHandlers]. Please consult documentation how to replace it accordingly.
   [junit4]   2> 91931 T447 n:127.0.0.1:52498_jn%2Fbc x:collection1 oasc.RequestHandlers.initHandlersFromConfig Registered paths: /admin/mbeans,standard,/update/csv,/update/json/docs,/admin/luke,/admin/segments,/get,/admin/system,/replication,/admin/properties,/config,/schema,/admin/plugins,/admin/logging,/update/json,/admin,/admin/threads,/admin/ping,/cdcr,/update,/admin/file
   [junit4]   2> 91931 T447 n:127.0.0.1:52498_jn%2Fbc x:collection1 oasc.SolrCore.initStatsCache Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 91932 T447 n:127.0.0.1:52498_jn%2Fbc x:collection1 oasu.UpdateHandler.<init> Using UpdateLog implementation: org.apache.solr.update.CdcrUpdateLog
   [junit4]   2> 91932 T447 n:127.0.0.1:52498_jn%2Fbc x:collection1 oasu.UpdateLog.init Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 91934 T447 n:127.0.0.1:52498_jn%2Fbc x:collection1 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 91934 T447 n:127.0.0.1:52498_jn%2Fbc x:collection1 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 91936 T447 n:127.0.0.1:52498_jn%2Fbc x:collection1 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=37, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
   [junit4]   2> 91936 T447 n:127.0.0.1:52498_jn%2Fbc x:collection1 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@2f6ee954 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@4109d5b3),segFN=segments_1,generation=1}
   [junit4]   2> 91936 T447 n:127.0.0.1:52498_jn%2Fbc x:collection1 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 91936 T447 n:127.0.0.1:52498_jn%2Fbc x:collection1 oass.SolrIndexSearcher.<init> Opening Searcher@3e6f1905[collection1] main
   [junit4]   2> 91937 T447 n:127.0.0.1:52498_jn%2Fbc x:collection1 oasu.UpdateLog.onFirstSearcher On first searcher opened, looking up max value of version field
   [junit4]   2> 91937 T447 n:127.0.0.1:52498_jn%2Fbc x:collection1 oasu.VersionInfo.getMaxVersionFromIndex Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 91937 T447 n:127.0.0.1:52498_jn%2Fbc x:collection1 oasu.VersionInfo.getMaxVersionFromIndex WARN No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 91937 T447 n:127.0.0.1:52498_jn%2Fbc x:collection1 oasu.UpdateLog.seedBucketsWithHighestVersion WARN Could not find max version in index or recent updates, using new clock 1501935497714335744
   [junit4]   2> 91938 T447 n:127.0.0.1:52498_jn%2Fbc x:collection1 oasu.UpdateLog.seedBucketsWithHighestVersion Took 1 ms to seed version buckets with highest version 1501935497714335744
   [junit4]   2> 91941 T447 n:127.0.0.1:52498_jn%2Fbc x:collection1 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 91941 T447 n:127.0.0.1:52498_jn%2Fbc x:collection1 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 91941 T447 n:127.0.0.1:52498_jn%2Fbc x:collection1 oasr.RestManager.init Initializing RestManager with initArgs: {}
   [junit4]   2> 91941 T447 n:127.0.0.1:52498_jn%2Fbc x:collection1 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 91944 T447 n:127.0.0.1:52498_jn%2Fbc x:collection1 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 91944 T447 n:127.0.0.1:52498_jn%2Fbc x:collection1 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 91944 T447 n:127.0.0.1:52498_jn%2Fbc x:collection1 oasr.RestManager.init Initializing 0 registered ManagedResources
   [junit4]   2> 91962 T447 n:127.0.0.1:52498_jn%2Fbc x:collection1 oasha.AdminHandlers.inform WARN <requestHandler name="/admin/" 
   [junit4]   2> 	 class="solr.admin.AdminHandlers" /> is deprecated . It is not required anymore
   [junit4]   2> 91963 T447 n:127.0.0.1:52498_jn%2Fbc x:collection1 oasc.ZkController.getConfDirListeners watch zkdir /configs/conf1
   [junit4]   2> 91964 T448 n:127.0.0.1:52498_jn%2Fbc x:collection1 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3e6f1905[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 91965 T447 n:127.0.0.1:52498_jn%2Fbc x:collection1 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 91966 T392 n:127.0.0.1:52498_jn%2Fbc oass.SolrDispatchFilter.init user.dir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J1
   [junit4]   2> 91966 T392 n:127.0.0.1:52498_jn%2Fbc oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 91997 T452 n:127.0.0.1:52498_jn%2Fbc c:tmp_collection s:shard1 x:collection1 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:52498/jn/bc collection:tmp_collection shard:shard1
   [junit4]   2> 92000 T452 n:127.0.0.1:52498_jn%2Fbc c:tmp_collection s:shard1 x:collection1 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
   [junit4]   2> 92001 T452 n:127.0.0.1:52498_jn%2Fbc c:tmp_collection s:shard1 x:collection1 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
   [junit4]   2> 92001 T452 n:127.0.0.1:52498_jn%2Fbc c:tmp_collection s:shard1 x:collection1 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
   [junit4]   2> ASYNC  NEW_CORE C83 name=collection1 org.apache.solr.core.SolrCore@4235f584 url=http://127.0.0.1:52498/jn/bc/collection1 node=127.0.0.1:52498_jn%2Fbc C83_STATE=coll:tmp_collection core:collection1 props:{core=collection1, base_url=http://127.0.0.1:52498/jn/bc, node_name=127.0.0.1:52498_jn%2Fbc, state=down}
   [junit4]   2> 92002 T452 n:127.0.0.1:52498_jn%2Fbc c:tmp_collection s:shard1 x:collection1 C83 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:52498/jn/bc/collection1/
   [junit4]   2> 92002 T452 n:127.0.0.1:52498_jn%2Fbc c:tmp_collection s:shard1 x:collection1 C83 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
   [junit4]   2> 92002 T452 n:127.0.0.1:52498_jn%2Fbc c:tmp_collection s:shard1 x:collection1 C83 oasc.SyncStrategy.syncToMe http://127.0.0.1:52498/jn/bc/collection1/ has no replicas
   [junit4]   2> 92002 T452 n:127.0.0.1:52498_jn%2Fbc c:tmp_collection s:shard1 x:collection1 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:52498/jn/bc/collection1/ shard1
   [junit4]   2> 92003 T446 n:127.0.0.1:52498_jn%2Fbc oash.CdcrLeaderStateManager$LeaderStateWatcher.process Received new leader state @ tmp_collection:shard1
   [junit4]   2> 92011 T420 n:127.0.0.1:52489_jn%2Fbc oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 92012 T421 n:127.0.0.1:52489_jn%2Fbc oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 2, message = {
   [junit4]   2> 	  "operation":"leader",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "collection":"tmp_collection"} current state version: 4
   [junit4]   2> 92014 T421 n:127.0.0.1:52489_jn%2Fbc oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "operation":"leader",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "collection":"tmp_collection",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:52498/jn/bc",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "state":"active"} current state version: 4
   [junit4]   2> 92080 T420 n:127.0.0.1:52489_jn%2Fbc oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 92134 T452 n:127.0.0.1:52498_jn%2Fbc c:tmp_collection s:shard1 x:collection1 oasc.ZkController.register We are http://127.0.0.1:52498/jn/bc/collection1/ and leader is http://127.0.0.1:52498/jn/bc/collection1/
   [junit4]   2> 92134 T452 n:127.0.0.1:52498_jn%2Fbc c:tmp_collection s:shard1 x:collection1 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:52498/jn/bc
   [junit4]   2> 92135 T452 n:127.0.0.1:52498_jn%2Fbc c:tmp_collection s:shard1 x:collection1 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
   [junit4]   2> 92135 T452 n:127.0.0.1:52498_jn%2Fbc c:tmp_collection s:shard1 x:collection1 oasc.ZkController.publish publishing core=collection1 state=active collection=tmp_collection
   [junit4]   2> 92135 T452 n:127.0.0.1:52498_jn%2Fbc c:tmp_collection s:shard1 x:collection1 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 92140 T453 n:127.0.0.1:52489_jn%2Fbc oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 92140 T421 n:127.0.0.1:52489_jn%2Fbc oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "core_node_name":"core_node2",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"http://127.0.0.1:52498/jn/bc",
   [junit4]   2> 	  "node_name":"127.0.0.1:52498_jn%2Fbc",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "state":"active",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "collection":"tmp_collection",
   [junit4]   2> 	  "operation":"state"} current state version: 5
   [junit4]   2> 92142 T421 n:127.0.0.1:52489_jn%2Fbc oasco.ReplicaMutator.updateState Update state numShards=2 message={
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "core_node_name":"core_node2",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"http://127.0.0.1:52498/jn/bc",
   [junit4]   2> 	  "node_name":"127.0.0.1:52498_jn%2Fbc",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "state":"active",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "collection":"tmp_collection",
   [junit4]   2> 	  "operation":"state"}
   [junit4]   2> 92337 T392 oas.SolrTestCaseJ4.writeCoreProperties Writing core.properties file to C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CdcrRequestHandlerTest 46273C4DCB58A707-001\jetty-003\cores\collection1
   [junit4]   2> 92340 T392 oejs.Server.doStart jetty-9.2.10.v20150310
   [junit4]   2> 92346 T392 oejsh.ContextHandler.doStart Started o.e.j.s.ServletContextHandler@398f3b43{/jn/bc,null,AVAILABLE}
   [junit4]   2> 92347 T392 oejs.AbstractConnector.doStart Started ServerConnector@322d007f{HTTP/1.1}{127.0.0.1:52507}
   [junit4]   2> 92347 T392 oejs.Server.doStart Started @99248ms
   [junit4]   2> 92347 T392 oascse.JettySolrRunner$1.lifeCycleStarted Jetty properties: {shards=shard3, hostContext=/jn/bc, hostPort=52507, coreRootDirectory=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CdcrRequestHandlerTest 46273C4DCB58A707-001\jetty-003\cores}
   [junit4]   2> 92348 T392 oass.SolrDispatchFilter.init SolrDispatchFilter.init()sun.misc.Launcher$AppClassLoader@4e0e2f2a
   [junit4]   2> 92348 T392 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CdcrRequestHandlerTest 46273C4DCB58A707-001\jetty-003\'
   [junit4]   2> 92370 T392 oasc.SolrXmlConfig.fromFile Loading container configuration from C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CdcrRequestHandlerTest 46273C4DCB58A707-001\jetty-003\solr.xml
   [junit4]   2> 92377 T392 oasc.CorePropertiesLocator.<init> Config-defined core root directory: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CdcrRequestHandlerTest 46273C4DCB58A707-001\jetty-003\cores
   [junit4]   2> 92378 T392 oasc.CoreContainer.<init> New CoreContainer 943447702
   [junit4]   2> 92378 T392 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CdcrRequestHandlerTest 46273C4DCB58A707-001\jetty-003\]
   [junit4]   2> 92378 T392 oasc.CoreContainer.load loading shared library: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CdcrRequestHandlerTest 46273C4DCB58A707-001\jetty-003\lib
   [junit4]   2> 92379 T392 oasc.SolrResourceLoader.addToClassLoader WARN Can't find (or read) directory to add to classloader: lib (resolved as: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CdcrRequestHandlerTest 46273C4DCB58A707-001\jetty-003\lib).
   [junit4]   2> 92389 T392 oashc.HttpShardHandlerFactory.init created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 92391 T392 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 92392 T392 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 92392 T392 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 92392 T392 oasc.CoreContainer.load Node Name: 127.0.0.1
   [junit4]   2> 92392 T392 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:52482/solr
   [junit4]   2> 92393 T392 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 93360 T392 n:127.0.0.1:52507_jn%2Fbc oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:52507_jn%2Fbc
   [junit4]   2> 93366 T392 n:127.0.0.1:52507_jn%2Fbc oasc.Overseer.close Overseer (id=null) closing
   [junit4]   2> 93370 T392 n:127.0.0.1:52507_jn%2Fbc oasc.CoreContainer.initializeAuthenticationPlugin No authentication plugin used.
   [junit4]   2> 93370 T392 n:127.0.0.1:52507_jn%2Fbc oasc.CoreContainer.intializeAuthorizationPlugin Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 93374 T392 n:127.0.0.1:52507_jn%2Fbc oasc.CorePropertiesLocator.discover Looking for core definitions underneath C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CdcrRequestHandlerTest 46273C4DCB58A707-001\jetty-003\cores
   [junit4]   2> 93376 T392 n:127.0.0.1:52507_jn%2Fbc oasc.CoreDescriptor.<init> CORE DESCRIPTOR: {name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, loadOnStartup=true, instanceDir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CdcrRequestHandlerTest 46273C4DCB58A707-001\jetty-003\cores\collection1, collection=tmp_collection, absoluteInstDir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CdcrRequestHandlerTest 46273C4DCB58A707-001\jetty-003\cores\collection1\, coreNodeName=, dataDir=data\, shard=}
   [junit4]   2> 93377 T392 n:127.0.0.1:52507_jn%2Fbc oasc.CorePropertiesLocator.discoverUnder Found core collection1 in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CdcrRequestHandlerTest 46273C4DCB58A707-001\jetty-003\cores\collection1\
   [junit4]   2> 93377 T392 n:127.0.0.1:52507_jn%2Fbc oasc.CorePropertiesLocator.discover Found 1 core definitions
   [junit4]   2> 93381 T471 n:127.0.0.1:52507_jn%2Fbc c:tmp_collection x:collection1 oasc.ZkController.publish publishing core=collection1 state=down collection=tmp_collection
   [junit4]   2> 93382 T471 n:127.0.0.1:52507_jn%2Fbc c:tmp_collection x:collection1 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 93383 T471 n:127.0.0.1:52507_jn%2Fbc oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 93384 T453 n:127.0.0.1:52489_jn%2Fbc oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 93386 T421 n:127.0.0.1:52489_jn%2Fbc oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"http://127.0.0.1:52507/jn/bc",
   [junit4]   2> 	  "node_name":"127.0.0.1:52507_jn%2Fbc",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "collection":"tmp_collection",
   [junit4]   2> 	  "operation":"state"} current state version: 6
   [junit4]   2> 93387 T421 n:127.0.0.1:52489_jn%2Fbc oasco.ReplicaMutator.updateState Update state numShards=2 message={
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"http://127.0.0.1:52507/jn/bc",
   [junit4]   2> 	  "node_name":"127.0.0.1:52507_jn%2Fbc",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "collection":"tmp_collection",
   [junit4]   2> 	  "operation":"state"}
   [junit4]   2> 93387 T421 n:127.0.0.1:52489_jn%2Fbc oasco.ReplicaMutator.updateState Collection already exists with numShards=2
   [junit4]   2> 93387 T421 n:127.0.0.1:52489_jn%2Fbc oasco.ReplicaMutator.updateState Assigning new node to shard shard=shard2
   [junit4]   2> 94327 T471 n:127.0.0.1:52507_jn%2Fbc oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
   [junit4]   2> 94327 T471 n:127.0.0.1:52507_jn%2Fbc oasc.ZkController.createCollectionZkNode Check for collection zkNode:tmp_collection
   [junit4]   2> 94328 T471 n:127.0.0.1:52507_jn%2Fbc oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 94329 T471 n:127.0.0.1:52507_jn%2Fbc oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CdcrRequestHandlerTest 46273C4DCB58A707-001\jetty-003\cores\collection1\'
   [junit4]   2> 94345 T471 n:127.0.0.1:52507_jn%2Fbc oasc.Config.<init> loaded config solrconfig.xml with version 0 
   [junit4]   2> 94352 T471 n:127.0.0.1:52507_jn%2Fbc oasc.SolrConfig.refreshRequestParams current version of requestparams : -1
   [junit4]   2> 94355 T471 n:127.0.0.1:52507_jn%2Fbc oasc.SolrConfig.<init> Using Lucene MatchVersion: 6.0.0
   [junit4]   2> 94361 T471 n:127.0.0.1:52507_jn%2Fbc oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 94362 T471 n:127.0.0.1:52507_jn%2Fbc oass.IndexSchema.readSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 94366 T471 n:127.0.0.1:52507_jn%2Fbc oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 94435 T471 n:127.0.0.1:52507_jn%2Fbc oass.IndexSchema.readSchema default search field in schema is t

[...truncated too long message...]

solr-core\test\J0\temp\solr.cloud.CdcrReplicationHandlerTest 46273C4DCB58A707-001\jetty-002\cores\source_collection_shard1_replica2\data\index [CachedDir<<refCount=0;path=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.CdcrReplicationHandlerTest 46273C4DCB58A707-001\jetty-002\cores\source_collection_shard1_replica2\data\index;done=false>>]
   [junit4]   2> 1504763 T6684 c:source_collection s:shard1 x:source_collection_shard1_replica2 oasc.CachingDirectoryFactory.close Closing directory: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.CdcrReplicationHandlerTest 46273C4DCB58A707-001\jetty-002\cores\source_collection_shard1_replica2\data\index
   [junit4]   2> 1504763 T6684 c:source_collection s:shard1 x:source_collection_shard1_replica2 oasc.Overseer.close Overseer (id=93871036725198890-127.0.0.1:55142_ny_m-n_0000000003) closing
   [junit4]   2> 1504763 T6972 n:127.0.0.1:55142_ny_m oasc.Overseer$ClusterStateUpdater.run Overseer Loop exiting : 127.0.0.1:55142_ny_m
   [junit4]   2> 1504766 T6968 n:127.0.0.1:55142_ny_m oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
   [junit4]   2> 1504767 T6684 oejs.AbstractConnector.doStop Stopped ServerConnector@7f2e5908{HTTP/1.1}{127.0.0.1:55142}
   [junit4]   2> 1504768 T6684 oejsh.ContextHandler.doStop Stopped o.e.j.s.ServletContextHandler@124b4890{/ny_m,null,UNAVAILABLE}
   [junit4]   2> 1504768 T6684 c:source_collection s:shard1 x:source_collection_shard1_replica2 oas.SolrTestCaseJ4.tearDown ###Ending doTest
   [junit4]   2> 1504768 T6684 c:source_collection s:shard1 x:source_collection_shard1_replica2 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:55126 55126
   [junit4]   2> 1506715 T6685 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:55126 55126
   [junit4]   2> 1506716 T6685 oasc.ZkTestServer$ZKServerMain.runFromConfig WARN Watch limit violations: 
   [junit4]   2> 	Maximum concurrent create/delete watches above limit:
   [junit4]   2> 	
   [junit4]   2> 		37	/solr/aliases.json
   [junit4]   2> 		37	/solr/clusterstate.json
   [junit4]   2> 		8	/solr/configs/conf1
   [junit4]   2> 		5	/solr/collections/source_collection/state.json
   [junit4]   2> 		4	/solr/collections/source_collection/leaders/shard1
   [junit4]   2> 		2	/solr/collections/tmp_collection/leaders/shard1
   [junit4]   2> 	
   [junit4]   2> 	Maximum concurrent data watches above limit:
   [junit4]   2> 	
   [junit4]   2> 		6	/solr/collections/source_collection/cdcr/state/buffer
   [junit4]   2> 		6	/solr/collections/source_collection/cdcr/state/process
   [junit4]   2> 		2	/solr/collections/source_collection/leader_elect/shard1/election/93871036725198853-core_node2-n_0000000000
   [junit4]   2> 		2	/solr/collections/tmp_collection/cdcr/state/buffer
   [junit4]   2> 		2	/solr/collections/source_collection/leader_elect/shard1/election/93871036725198871-core_node1-n_0000000000
   [junit4]   2> 		2	/solr/collections/tmp_collection/cdcr/state/process
   [junit4]   2> 	
   [junit4]   2> 	Maximum concurrent children watches above limit:
   [junit4]   2> 	
   [junit4]   2> 		35	/solr/live_nodes
   [junit4]   2> 		4	/solr/overseer/queue
   [junit4]   2> 		4	/solr/overseer/collection-queue-work
   [junit4]   2> 	
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=CdcrReplicationHandlerTest -Dtests.method=doTest -Dtests.seed=46273C4DCB58A707 -Dtests.slow=true -Dtests.locale=tr -Dtests.timezone=Africa/Libreville -Dtests.asserts=true -Dtests.file.encoding=Cp1252
   [junit4] FAILURE  375s J0 | CdcrReplicationHandlerTest.doTest <<<
   [junit4]    > Throwable #1: java.lang.AssertionError: There are still nodes recoverying - waited for 330 seconds
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([46273C4DCB58A707:E16384E9A6E3B4BE]:0)
   [junit4]    > 	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:172)
   [junit4]    > 	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:133)
   [junit4]    > 	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:128)
   [junit4]    > 	at org.apache.solr.cloud.BaseCdcrDistributedZkTest.waitForRecoveriesToFinish(BaseCdcrDistributedZkTest.java:419)
   [junit4]    > 	at org.apache.solr.cloud.BaseCdcrDistributedZkTest.restartServer(BaseCdcrDistributedZkTest.java:460)
   [junit4]    > 	at org.apache.solr.cloud.CdcrReplicationHandlerTest.doTestPartialReplication(CdcrReplicationHandlerTest.java:109)
   [junit4]    > 	at org.apache.solr.cloud.CdcrReplicationHandlerTest.doTest(CdcrReplicationHandlerTest.java:51)
   [junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:960)
   [junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:935)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 1506721 T6684 c:source_collection s:shard1 x:source_collection_shard1_replica2 oas.SolrTestCaseJ4.deleteCore ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.CdcrReplicationHandlerTest 46273C4DCB58A707-001
   [junit4]   2> NOTE: test params are: codec=CheapBastard, sim=RandomSimilarityProvider(queryNorm=true,coord=yes): {}, locale=tr, timezone=Africa/Libreville
   [junit4]   2> NOTE: Windows 7 6.1 amd64/Oracle Corporation 1.8.0_45 (64-bit)/cpus=3,threads=1,free=246523832,total=536870912
   [junit4]   2> NOTE: All tests run in this JVM: [SliceStateTest, UniqFieldsUpdateProcessorFactoryTest, TestDistributedSearch, TestPostingsSolrHighlighter, CollectionReloadTest, SolrTestCaseJ4Test, JSONWriterTest, TestRestManager, SimpleFacetsTest, PathHierarchyTokenizerFactoryTest, ConcurrentDeleteAndCreateCollectionTest, SOLR749Test, TestStressReorder, TestSchemaVersionResource, TestFileDictionaryLookup, TestCollationField, TestIBSimilarityFactory, SampleTest, TestDocumentBuilder, TestMiniSolrCloudClusterSSL, TestCryptoKeys, TestOrdValues, TestManagedSchemaFieldResource, TestXIncludeConfig, RuleEngineTest, SharedFSAutoReplicaFailoverTest, TestShardHandlerFactory, TestDynamicLoading, TestComponentsName, LeaderFailoverAfterPartitionTest, TestSchemaResource, TestCursorMarkWithoutUniqueKey, SoftAutoCommitTest, PeerSyncTest, DistributedQueryElevationComponentTest, TestDefaultStatsCache, OverseerTest, TestLeaderElectionZkExpiry, TestConfigSets, TestPivotHelperCode, LeaderElectionTest, TestUpdate, UUIDFieldTest, DistanceUnitsTest, InfoHandlerTest, CircularListTest, PrimUtilsTest, SynonymTokenizerTest, DeleteInactiveReplicaTest, TestReplicationHandler, TestStressLucene, TestDocBasedVersionConstraints, CollectionsAPIAsyncDistributedZkTest, HdfsNNFailoverTest, TestCoreContainer, TestFieldResource, TestMacros, TestAnalyzedSuggestions, ConnectionManagerTest, CoreAdminRequestStatusTest, DOMUtilTest, TestTrackingShardHandlerFactory, TestStressVersions, RemoteQueryErrorTest, DistribDocExpirationUpdateProcessorTest, TestFastLRUCache, TestMaxScoreQueryParser, IgnoreCommitOptimizeUpdateProcessorFactoryTest, TestLRUCache, SolrIndexSplitterTest, SyncSliceTest, DeleteReplicaTest, HighlighterMaxOffsetTest, HdfsRecoveryZkTest, TestSweetSpotSimilarityFactory, XmlUpdateRequestHandlerTest, TestGroupingSearch, QueryResultKeyTest, ClusterStateTest, SSLMigrationTest, TestCollationFieldDocValues, TestCloudPivotFacet, TestChildDocTransformer, BadComponentTest, NoCacheHeaderTest, TestComplexPhraseQParserPlugin, StressHdfsTest, TestSolrXml, DistribCursorPagingTest, PreAnalyzedFieldTest, TestFreeTextSuggestions, AlternateDirectoryTest, TestIndexingPerformance, TestStressRecovery, HdfsSyncSliceTest, ClusterStateUpdateTest, DocValuesTest, TestIndexSearcher, TestArbitraryIndexDir, TestJmxMonitoredMap, TestJsonFacets, TimeZoneUtilsTest, TestDistributedMissingSort, TestRawResponseWriter, TestSolrQueryParserDefaultOperatorResource, IndexSchemaRuntimeFieldTest, SolrCoreTest, ResponseLogComponentTest, FacetPivotSmallTest, TestFaceting, DistributedIntervalFacetingTest, SolrCoreCheckLockOnStartupTest, DistributedFacetPivotWhiteBoxTest, TestHashQParserPlugin, CdcrReplicationHandlerTest]
   [junit4] Completed [495/502] on J0 in 375.44s, 1 test, 1 failure <<< FAILURES!

[...truncated 30 lines...]
BUILD FAILED
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\build.xml:526: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\build.xml:474: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\build.xml:61: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\extra-targets.xml:39: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build.xml:229: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\common-build.xml:512: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\common-build.xml:1415: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\common-build.xml:973: There were test failures: 502 suites, 1986 tests, 3 suite-level errors, 2 failures, 1475 ignored (20 assumptions)

Total time: 45 minutes 12 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-trunk-Windows (64bit/jdk1.8.0_45) - Build # 4842 - Still Failing!

Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Windows/4842/
Java: 64bit/jdk1.8.0_45 -XX:-UseCompressedOops -XX:+UseSerialGC

5 tests failed.
FAILED:  org.apache.solr.cloud.CdcrReplicationDistributedZkTest.doTest

Error Message:
expected:<st[art]ed> but was:<st[opp]ed>

Stack Trace:
org.junit.ComparisonFailure: expected:<st[art]ed> but was:<st[opp]ed>
	at __randomizedtesting.SeedInfo.seed([2C54B4B78A95EA2F:8B100C13E72EF996]:0)
	at org.junit.Assert.assertEquals(Assert.java:125)
	at org.junit.Assert.assertEquals(Assert.java:147)
	at org.apache.solr.cloud.BaseCdcrDistributedZkTest.assertState(BaseCdcrDistributedZkTest.java:255)
	at org.apache.solr.cloud.CdcrReplicationDistributedZkTest.doTestTargetCollectionNotAvailable(CdcrReplicationDistributedZkTest.java:113)
	at org.apache.solr.cloud.CdcrReplicationDistributedZkTest.doTest(CdcrReplicationDistributedZkTest.java:42)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:497)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:960)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:935)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.cloud.CdcrReplicationHandlerTest.doTest

Error Message:
There are still nodes recoverying - waited for 330 seconds

Stack Trace:
java.lang.AssertionError: There are still nodes recoverying - waited for 330 seconds
	at __randomizedtesting.SeedInfo.seed([2C54B4B78A95EA2F:8B100C13E72EF996]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:172)
	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:133)
	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:128)
	at org.apache.solr.cloud.BaseCdcrDistributedZkTest.waitForRecoveriesToFinish(BaseCdcrDistributedZkTest.java:419)
	at org.apache.solr.cloud.BaseCdcrDistributedZkTest.restartServer(BaseCdcrDistributedZkTest.java:460)
	at org.apache.solr.cloud.CdcrReplicationHandlerTest.doTestPartialReplication(CdcrReplicationHandlerTest.java:109)
	at org.apache.solr.cloud.CdcrReplicationHandlerTest.doTest(CdcrReplicationHandlerTest.java:51)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:497)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:960)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:935)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.cloud.CdcrRequestHandlerTest.doTest

Error Message:
expected:<st[art]ed> but was:<st[opp]ed>

Stack Trace:
org.junit.ComparisonFailure: expected:<st[art]ed> but was:<st[opp]ed>
	at __randomizedtesting.SeedInfo.seed([2C54B4B78A95EA2F:8B100C13E72EF996]:0)
	at org.junit.Assert.assertEquals(Assert.java:125)
	at org.junit.Assert.assertEquals(Assert.java:147)
	at org.apache.solr.cloud.BaseCdcrDistributedZkTest.assertState(BaseCdcrDistributedZkTest.java:255)
	at org.apache.solr.cloud.CdcrRequestHandlerTest.doTestLifeCycleActions(CdcrRequestHandlerTest.java:55)
	at org.apache.solr.cloud.CdcrRequestHandlerTest.doTest(CdcrRequestHandlerTest.java:39)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:497)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:960)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:935)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.update.processor.AtomicUpdatesTest.testRemove

Error Message:
Test abandoned because suite timeout was reached.

Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
	at __randomizedtesting.SeedInfo.seed([2C54B4B78A95EA2F]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.update.processor.AtomicUpdatesTest

Error Message:
Suite timeout exceeded (>= 7200000 msec).

Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
	at __randomizedtesting.SeedInfo.seed([2C54B4B78A95EA2F]:0)




Build Log:
[...truncated 9717 lines...]
   [junit4] Suite: org.apache.solr.cloud.CdcrRequestHandlerTest
   [junit4]   2> Creating dataDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CdcrRequestHandlerTest 2C54B4B78A95EA2F-001\init-core-data-001
   [junit4]   2> 998370 T6411 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (false) and clientAuth (false)
   [junit4]   2> 998371 T6411 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /loks/vq
   [junit4]   2> 998374 T6411 oasc.ZkTestServer.run STARTING ZK TEST SERVER
   [junit4]   2> 998374 T6412 oasc.ZkTestServer$2$1.setClientPort client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 998375 T6412 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
   [junit4]   2> 998463 T6411 oasc.ZkTestServer.run start zk server on port:63767
   [junit4]   2> 998463 T6411 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 998465 T6411 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 998469 T6419 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4a554d18 name:ZooKeeperConnection Watcher:127.0.0.1:63767 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 998469 T6411 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 998469 T6411 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 998470 T6411 oascc.SolrZkClient.makePath makePath: /solr
   [junit4]   2> 998476 T6411 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 998477 T6411 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 998479 T6422 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@69de5895 name:ZooKeeperConnection Watcher:127.0.0.1:63767/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 998479 T6411 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 998479 T6411 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 998479 T6411 oascc.SolrZkClient.makePath makePath: /collections/collection1
   [junit4]   2> 998482 T6411 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
   [junit4]   2> 998484 T6411 oascc.SolrZkClient.makePath makePath: /collections/control_collection
   [junit4]   2> 998485 T6411 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
   [junit4]   2> 998488 T6411 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\solrconfig-cdcr.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 998488 T6411 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 998492 T6411 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\schema15.xml to /configs/conf1/schema.xml
   [junit4]   2> 998492 T6411 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
   [junit4]   2> 998495 T6411 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 998495 T6411 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 998498 T6411 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 998498 T6411 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 998500 T6411 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 998500 T6411 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
   [junit4]   2> 998502 T6411 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 998503 T6411 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
   [junit4]   2> 998504 T6411 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 998505 T6411 oascc.SolrZkClient.makePath makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 998507 T6411 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 998507 T6411 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 998510 T6411 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 998510 T6411 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 998514 T6411 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 998515 T6411 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 998517 T6411 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 998517 T6411 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 998520 T6411 oas.SolrTestCaseJ4.setUp ###Starting doTest
   [junit4]   2> 998784 T6411 oas.SolrTestCaseJ4.writeCoreProperties Writing core.properties file to C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CdcrRequestHandlerTest 2C54B4B78A95EA2F-001\jetty-001\cores\collection1
   [junit4]   2> 998787 T6411 oejs.Server.doStart jetty-9.2.10.v20150310
   [junit4]   2> 998789 T6411 oejsh.ContextHandler.doStart Started o.e.j.s.ServletContextHandler@2e1fbaea{/loks/vq,null,AVAILABLE}
   [junit4]   2> 998790 T6411 oejs.AbstractConnector.doStart Started ServerConnector@5e83ad44{HTTP/1.1}{127.0.0.1:63774}
   [junit4]   2> 998790 T6411 oejs.Server.doStart Started @999508ms
   [junit4]   2> 998790 T6411 oascse.JettySolrRunner$1.lifeCycleStarted Jetty properties: {shards=shard1, hostContext=/loks/vq, hostPort=63774, coreRootDirectory=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CdcrRequestHandlerTest 2C54B4B78A95EA2F-001\jetty-001\cores}
   [junit4]   2> 998791 T6411 oass.SolrDispatchFilter.init SolrDispatchFilter.init()sun.misc.Launcher$AppClassLoader@4e0e2f2a
   [junit4]   2> 998791 T6411 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CdcrRequestHandlerTest 2C54B4B78A95EA2F-001\jetty-001\'
   [junit4]   2> 998813 T6411 oasc.SolrXmlConfig.fromFile Loading container configuration from C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CdcrRequestHandlerTest 2C54B4B78A95EA2F-001\jetty-001\solr.xml
   [junit4]   2> 998818 T6411 oasc.CorePropertiesLocator.<init> Config-defined core root directory: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CdcrRequestHandlerTest 2C54B4B78A95EA2F-001\jetty-001\cores
   [junit4]   2> 998819 T6411 oasc.CoreContainer.<init> New CoreContainer 1589180171
   [junit4]   2> 998819 T6411 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CdcrRequestHandlerTest 2C54B4B78A95EA2F-001\jetty-001\]
   [junit4]   2> 998819 T6411 oasc.CoreContainer.load loading shared library: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CdcrRequestHandlerTest 2C54B4B78A95EA2F-001\jetty-001\lib
   [junit4]   2> 998820 T6411 oasc.SolrResourceLoader.addToClassLoader WARN Can't find (or read) directory to add to classloader: lib (resolved as: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CdcrRequestHandlerTest 2C54B4B78A95EA2F-001\jetty-001\lib).
   [junit4]   2> 998826 T6411 oashc.HttpShardHandlerFactory.init created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 998828 T6411 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 998828 T6411 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 998829 T6411 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 998829 T6411 oasc.CoreContainer.load Node Name: 127.0.0.1
   [junit4]   2> 998830 T6411 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:63767/solr
   [junit4]   2> 998830 T6411 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 998830 T6411 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 998832 T6411 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 998834 T6436 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5e9d6a33 name:ZooKeeperConnection Watcher:127.0.0.1:63767 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 998834 T6411 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 998834 T6411 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 998837 T6411 n:127.0.0.1:63774_loks%2Fvq oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 998839 T6439 n:127.0.0.1:63774_loks%2Fvq oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@24c7744f name:ZooKeeperConnection Watcher:127.0.0.1:63767/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 998839 T6411 n:127.0.0.1:63774_loks%2Fvq oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 998840 T6411 n:127.0.0.1:63774_loks%2Fvq oascc.SolrZkClient.makePath makePath: /overseer/queue
   [junit4]   2> 998842 T6411 n:127.0.0.1:63774_loks%2Fvq oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
   [junit4]   2> 998844 T6411 n:127.0.0.1:63774_loks%2Fvq oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
   [junit4]   2> 998846 T6411 n:127.0.0.1:63774_loks%2Fvq oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
   [junit4]   2> 998848 T6411 n:127.0.0.1:63774_loks%2Fvq oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
   [junit4]   2> 998851 T6411 n:127.0.0.1:63774_loks%2Fvq oascc.SolrZkClient.makePath makePath: /live_nodes
   [junit4]   2> 998852 T6411 n:127.0.0.1:63774_loks%2Fvq oascc.SolrZkClient.makePath makePath: /aliases.json
   [junit4]   2> 998853 T6411 n:127.0.0.1:63774_loks%2Fvq oascc.SolrZkClient.makePath makePath: /clusterstate.json
   [junit4]   2> 998855 T6411 n:127.0.0.1:63774_loks%2Fvq oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:63774_loks%2Fvq
   [junit4]   2> 998855 T6411 n:127.0.0.1:63774_loks%2Fvq oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:63774_loks%2Fvq
   [junit4]   2> 998857 T6411 n:127.0.0.1:63774_loks%2Fvq oascc.SolrZkClient.makePath makePath: /overseer_elect
   [junit4]   2> 998859 T6411 n:127.0.0.1:63774_loks%2Fvq oascc.SolrZkClient.makePath makePath: /overseer_elect/election
   [junit4]   2> 998861 T6411 n:127.0.0.1:63774_loks%2Fvq oasc.Overseer.close Overseer (id=null) closing
   [junit4]   2> 998862 T6411 n:127.0.0.1:63774_loks%2Fvq oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:63774_loks%2Fvq
   [junit4]   2> 998863 T6411 n:127.0.0.1:63774_loks%2Fvq oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
   [junit4]   2> 998864 T6411 n:127.0.0.1:63774_loks%2Fvq oasc.Overseer.start Overseer (id=93869284534255619-127.0.0.1:63774_loks%2Fvq-n_0000000000) starting
   [junit4]   2> 998866 T6411 n:127.0.0.1:63774_loks%2Fvq oascc.SolrZkClient.makePath makePath: /overseer/queue-work
   [junit4]   2> 998873 T6411 n:127.0.0.1:63774_loks%2Fvq oasc.OverseerAutoReplicaFailoverThread.<init> Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 998874 T6411 n:127.0.0.1:63774_loks%2Fvq oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 998874 T6441 n:127.0.0.1:63774_loks%2Fvq oasc.OverseerCollectionProcessor.run Process current queue of collection creations
   [junit4]   2> 998874 T6440 n:127.0.0.1:63774_loks%2Fvq oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
   [junit4]   2> 998877 T6411 n:127.0.0.1:63774_loks%2Fvq oasc.CoreContainer.initializeAuthenticationPlugin No authentication plugin used.
   [junit4]   2> 998877 T6411 n:127.0.0.1:63774_loks%2Fvq oasc.CoreContainer.intializeAuthorizationPlugin Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 998878 T6411 n:127.0.0.1:63774_loks%2Fvq oasc.CorePropertiesLocator.discover Looking for core definitions underneath C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CdcrRequestHandlerTest 2C54B4B78A95EA2F-001\jetty-001\cores
   [junit4]   2> 998880 T6411 n:127.0.0.1:63774_loks%2Fvq oasc.CoreDescriptor.<init> CORE DESCRIPTOR: {name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, loadOnStartup=true, instanceDir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CdcrRequestHandlerTest 2C54B4B78A95EA2F-001\jetty-001\cores\collection1, collection=tmp_collection, absoluteInstDir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CdcrRequestHandlerTest 2C54B4B78A95EA2F-001\jetty-001\cores\collection1\, coreNodeName=, dataDir=data\, shard=}
   [junit4]   2> 998880 T6411 n:127.0.0.1:63774_loks%2Fvq oasc.CorePropertiesLocator.discoverUnder Found core collection1 in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CdcrRequestHandlerTest 2C54B4B78A95EA2F-001\jetty-001\cores\collection1\
   [junit4]   2> 998880 T6411 n:127.0.0.1:63774_loks%2Fvq oasc.CorePropertiesLocator.discover Found 1 core definitions
   [junit4]   2> 998881 T6443 n:127.0.0.1:63774_loks%2Fvq c:tmp_collection x:collection1 oasc.ZkController.publish publishing core=collection1 state=down collection=tmp_collection
   [junit4]   2> 998881 T6443 n:127.0.0.1:63774_loks%2Fvq c:tmp_collection x:collection1 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 998882 T6439 n:127.0.0.1:63774_loks%2Fvq oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 998882 T6443 n:127.0.0.1:63774_loks%2Fvq oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 998883 T6440 n:127.0.0.1:63774_loks%2Fvq oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"http://127.0.0.1:63774/loks/vq",
   [junit4]   2> 	  "node_name":"127.0.0.1:63774_loks%2Fvq",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "collection":"tmp_collection",
   [junit4]   2> 	  "operation":"state"} current state version: 0
   [junit4]   2> 998884 T6440 n:127.0.0.1:63774_loks%2Fvq oasco.ReplicaMutator.updateState Update state numShards=2 message={
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"http://127.0.0.1:63774/loks/vq",
   [junit4]   2> 	  "node_name":"127.0.0.1:63774_loks%2Fvq",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "collection":"tmp_collection",
   [junit4]   2> 	  "operation":"state"}
   [junit4]   2> 998884 T6440 n:127.0.0.1:63774_loks%2Fvq oasco.ClusterStateMutator.createCollection building a new cName: tmp_collection
   [junit4]   2> 998884 T6440 n:127.0.0.1:63774_loks%2Fvq oasco.ReplicaMutator.updateState Assigning new node to shard shard=shard2
   [junit4]   2> 998885 T6439 n:127.0.0.1:63774_loks%2Fvq oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 999764 T6443 n:127.0.0.1:63774_loks%2Fvq oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
   [junit4]   2> 999764 T6443 n:127.0.0.1:63774_loks%2Fvq oasc.ZkController.createCollectionZkNode Check for collection zkNode:tmp_collection
   [junit4]   2> 999765 T6443 n:127.0.0.1:63774_loks%2Fvq oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:tmp_collection
   [junit4]   2> 999765 T6443 n:127.0.0.1:63774_loks%2Fvq oasc.ZkController.getConfName Looking for collection configName
   [junit4]   2> 999766 T6443 n:127.0.0.1:63774_loks%2Fvq oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
   [junit4]   2> 999766 T6443 n:127.0.0.1:63774_loks%2Fvq oascc.SolrZkClient.makePath makePath: /collections/tmp_collection
   [junit4]   2> 999767 T6443 n:127.0.0.1:63774_loks%2Fvq oascc.ZkStateReader.readConfigName Load collection config from:/collections/tmp_collection
   [junit4]   2> 999768 T6443 n:127.0.0.1:63774_loks%2Fvq oascc.ZkStateReader.readConfigName path=/collections/tmp_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 999768 T6443 n:127.0.0.1:63774_loks%2Fvq oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CdcrRequestHandlerTest 2C54B4B78A95EA2F-001\jetty-001\cores\collection1\'
   [junit4]   2> 999781 T6443 n:127.0.0.1:63774_loks%2Fvq oasc.Config.<init> loaded config solrconfig.xml with version 0 
   [junit4]   2> 999787 T6443 n:127.0.0.1:63774_loks%2Fvq oasc.SolrConfig.refreshRequestParams current version of requestparams : -1
   [junit4]   2> 999797 T6443 n:127.0.0.1:63774_loks%2Fvq oasc.SolrConfig.<init> Using Lucene MatchVersion: 6.0.0
   [junit4]   2> 999802 T6443 n:127.0.0.1:63774_loks%2Fvq oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 999804 T6443 n:127.0.0.1:63774_loks%2Fvq oass.IndexSchema.readSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 999807 T6443 n:127.0.0.1:63774_loks%2Fvq oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 999869 T6443 n:127.0.0.1:63774_loks%2Fvq oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 999870 T6443 n:127.0.0.1:63774_loks%2Fvq oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 999870 T6443 n:127.0.0.1:63774_loks%2Fvq oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 999873 T6443 n:127.0.0.1:63774_loks%2Fvq oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 999886 T6443 n:127.0.0.1:63774_loks%2Fvq oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection tmp_collection
   [junit4]   2> 999886 T6443 n:127.0.0.1:63774_loks%2Fvq x:collection1 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 999887 T6443 n:127.0.0.1:63774_loks%2Fvq x:collection1 oasc.SolrCore.<init> [[collection1] ] Opening new SolrCore at [C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CdcrRequestHandlerTest 2C54B4B78A95EA2F-001\jetty-001\cores\collection1\], dataDir=[null]
   [junit4]   2> 999887 T6443 n:127.0.0.1:63774_loks%2Fvq x:collection1 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@573d2c90
   [junit4]   2> 999887 T6443 n:127.0.0.1:63774_loks%2Fvq x:collection1 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CdcrRequestHandlerTest 2C54B4B78A95EA2F-001\jetty-001\cores\collection1\data\
   [junit4]   2> 999887 T6443 n:127.0.0.1:63774_loks%2Fvq x:collection1 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CdcrRequestHandlerTest 2C54B4B78A95EA2F-001\jetty-001\cores\collection1\data\index/
   [junit4]   2> 999887 T6443 n:127.0.0.1:63774_loks%2Fvq x:collection1 oasc.SolrCore.initIndex WARN [collection1] Solr index directory 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CdcrRequestHandlerTest 2C54B4B78A95EA2F-001\jetty-001\cores\collection1\data\index' doesn't exist. Creating new index...
   [junit4]   2> 999888 T6443 n:127.0.0.1:63774_loks%2Fvq x:collection1 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CdcrRequestHandlerTest 2C54B4B78A95EA2F-001\jetty-001\cores\collection1\data\index
   [junit4]   2> 999888 T6443 n:127.0.0.1:63774_loks%2Fvq x:collection1 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=35, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.8468594779792245]
   [junit4]   2> 999888 T6443 n:127.0.0.1:63774_loks%2Fvq x:collection1 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@6b64883f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@84a87dc),segFN=segments_1,generation=1}
   [junit4]   2> 999889 T6443 n:127.0.0.1:63774_loks%2Fvq x:collection1 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 999890 T6443 n:127.0.0.1:63774_loks%2Fvq x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "cdcr-processor-chain"
   [junit4]   2> 999916 T6443 n:127.0.0.1:63774_loks%2Fvq x:collection1 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 999919 T6443 n:127.0.0.1:63774_loks%2Fvq x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 999920 T6443 n:127.0.0.1:63774_loks%2Fvq x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 999920 T6443 n:127.0.0.1:63774_loks%2Fvq x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 999921 T6443 n:127.0.0.1:63774_loks%2Fvq x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 999922 T6443 n:127.0.0.1:63774_loks%2Fvq x:collection1 oasc.PluginBag.createPlugin requestHandler : '/replication' created with startup=lazy 
   [junit4]   2> 999947 T6443 n:127.0.0.1:63774_loks%2Fvq x:collection1 oasc.SolrResourceLoader.findClass WARN Solr loaded a deprecated plugin/analysis class [org.apache.solr.handler.admin.AdminHandlers]. Please consult documentation how to replace it accordingly.
   [junit4]   2> 999948 T6443 n:127.0.0.1:63774_loks%2Fvq x:collection1 oasc.SolrResourceLoader.findClass WARN Solr loaded a deprecated plugin/analysis class [org.apache.solr.handler.admin.AdminHandlers]. Please consult documentation how to replace it accordingly.
   [junit4]   2> 999948 T6443 n:127.0.0.1:63774_loks%2Fvq x:collection1 oasc.RequestHandlers.initHandlersFromConfig Registered paths: /admin/mbeans,standard,/update/csv,/update/json/docs,/admin/luke,/admin/segments,/get,/admin/system,/replication,/admin/properties,/config,/schema,/admin/plugins,/admin/logging,/update/json,/admin,/admin/threads,/admin/ping,/cdcr,/update,/admin/file
   [junit4]   2> 999948 T6443 n:127.0.0.1:63774_loks%2Fvq x:collection1 oasc.SolrCore.initStatsCache Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 999966 T6443 n:127.0.0.1:63774_loks%2Fvq x:collection1 oasu.UpdateHandler.<init> Using UpdateLog implementation: org.apache.solr.update.CdcrUpdateLog
   [junit4]   2> 999966 T6443 n:127.0.0.1:63774_loks%2Fvq x:collection1 oasu.UpdateLog.init Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 999967 T6443 n:127.0.0.1:63774_loks%2Fvq x:collection1 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 999967 T6443 n:127.0.0.1:63774_loks%2Fvq x:collection1 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 999967 T6443 n:127.0.0.1:63774_loks%2Fvq x:collection1 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=34, maxMergeAtOnceExplicit=20, maxMergedSegmentMB=44.330078125, floorSegmentMB=0.318359375, forceMergeDeletesPctAllowed=25.688269165098806, segmentsPerTier=21.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
   [junit4]   2> 999968 T6443 n:127.0.0.1:63774_loks%2Fvq x:collection1 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@6b64883f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@84a87dc),segFN=segments_1,generation=1}
   [junit4]   2> 999968 T6443 n:127.0.0.1:63774_loks%2Fvq x:collection1 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 999968 T6443 n:127.0.0.1:63774_loks%2Fvq x:collection1 oass.SolrIndexSearcher.<init> Opening Searcher@3e2d8473[collection1] main
   [junit4]   2> 999968 T6443 n:127.0.0.1:63774_loks%2Fvq x:collection1 oasu.UpdateLog.onFirstSearcher On first searcher opened, looking up max value of version field
   [junit4]   2> 999968 T6443 n:127.0.0.1:63774_loks%2Fvq x:collection1 oasu.VersionInfo.getMaxVersionFromIndex Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 999968 T6443 n:127.0.0.1:63774_loks%2Fvq x:collection1 oasu.VersionInfo.getMaxVersionFromIndex WARN No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 999968 T6443 n:127.0.0.1:63774_loks%2Fvq x:collection1 oasu.UpdateLog.seedBucketsWithHighestVersion WARN Could not find max version in index or recent updates, using new clock 1501908554215325696
   [junit4]   2> 999970 T6443 n:127.0.0.1:63774_loks%2Fvq x:collection1 oasu.UpdateLog.seedBucketsWithHighestVersion Took 2 ms to seed version buckets with highest version 1501908554215325696
   [junit4]   2> 999970 T6443 n:127.0.0.1:63774_loks%2Fvq x:collection1 oascc.ZkStateReader.readConfigName Load collection config from:/collections/tmp_collection
   [junit4]   2> 999971 T6443 n:127.0.0.1:63774_loks%2Fvq x:collection1 oascc.ZkStateReader.readConfigName path=/collections/tmp_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 999971 T6443 n:127.0.0.1:63774_loks%2Fvq x:collection1 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 999972 T6443 n:127.0.0.1:63774_loks%2Fvq x:collection1 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 999972 T6443 n:127.0.0.1:63774_loks%2Fvq x:collection1 oasr.RestManager.init Initializing RestManager with initArgs: {}
   [junit4]   2> 999972 T6443 n:127.0.0.1:63774_loks%2Fvq x:collection1 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 999972 T6443 n:127.0.0.1:63774_loks%2Fvq x:collection1 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 999972 T6443 n:127.0.0.1:63774_loks%2Fvq x:collection1 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 999972 T6443 n:127.0.0.1:63774_loks%2Fvq x:collection1 oasr.RestManager.init Initializing 0 registered ManagedResources
   [junit4]   2> 999977 T6443 n:127.0.0.1:63774_loks%2Fvq x:collection1 oascc.SolrZkClient.makePath makePath: /collections/tmp_collection/cdcr/state
   [junit4]   2> 999980 T6443 n:127.0.0.1:63774_loks%2Fvq x:collection1 oash.CdcrBufferStateManager.createStateNode Created znode /collections/tmp_collection/cdcr/state/buffer
   [junit4]   2> 999986 T6443 n:127.0.0.1:63774_loks%2Fvq x:collection1 oash.CdcrProcessStateManager.createStateNode Created znode /collections/tmp_collection/cdcr/state/process
   [junit4]   2> 1000012 T6443 n:127.0.0.1:63774_loks%2Fvq x:collection1 oasha.AdminHandlers.inform WARN <requestHandler name="/admin/" 
   [junit4]   2> 	 class="solr.admin.AdminHandlers" /> is deprecated . It is not required anymore
   [junit4]   2> 1000013 T6443 n:127.0.0.1:63774_loks%2Fvq x:collection1 oasc.ZkController.getConfDirListeners watch zkdir /configs/conf1
   [junit4]   2> 1000013 T6444 n:127.0.0.1:63774_loks%2Fvq x:collection1 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3e2d8473[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1000013 T6443 n:127.0.0.1:63774_loks%2Fvq x:collection1 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 1000014 T6448 n:127.0.0.1:63774_loks%2Fvq c:tmp_collection s:shard2 x:collection1 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:63774/loks/vq collection:tmp_collection shard:shard2
   [junit4]   2> 1000014 T6411 n:127.0.0.1:63774_loks%2Fvq oass.SolrDispatchFilter.init user.dir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J1
   [junit4]   2> 1000014 T6411 n:127.0.0.1:63774_loks%2Fvq oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 1000015 T6448 n:127.0.0.1:63774_loks%2Fvq c:tmp_collection s:shard2 x:collection1 oascc.SolrZkClient.makePath makePath: /collections/tmp_collection/leader_elect/shard2/election
   [junit4]   2> 1000020 T6448 n:127.0.0.1:63774_loks%2Fvq c:tmp_collection s:shard2 x:collection1 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard2
   [junit4]   2> 1000021 T6439 n:127.0.0.1:63774_loks%2Fvq oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 1000021 T6448 n:127.0.0.1:63774_loks%2Fvq c:tmp_collection s:shard2 x:collection1 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
   [junit4]   2> 1000021 T6448 n:127.0.0.1:63774_loks%2Fvq c:tmp_collection s:shard2 x:collection1 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
   [junit4]   2> ASYNC  NEW_CORE C2884 name=collection1 org.apache.solr.core.SolrCore@1e09411b url=http://127.0.0.1:63774/loks/vq/collection1 node=127.0.0.1:63774_loks%2Fvq C2884_STATE=coll:tmp_collection core:collection1 props:{core=collection1, base_url=http://127.0.0.1:63774/loks/vq, node_name=127.0.0.1:63774_loks%2Fvq, state=down}
   [junit4]   2> 1000021 T6448 n:127.0.0.1:63774_loks%2Fvq c:tmp_collection s:shard2 x:collection1 C2884 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:63774/loks/vq/collection1/
   [junit4]   2> 1000022 T6448 n:127.0.0.1:63774_loks%2Fvq c:tmp_collection s:shard2 x:collection1 C2884 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
   [junit4]   2> 1000022 T6448 n:127.0.0.1:63774_loks%2Fvq c:tmp_collection s:shard2 x:collection1 C2884 oasc.SyncStrategy.syncToMe http://127.0.0.1:63774/loks/vq/collection1/ has no replicas
   [junit4]   2> 1000022 T6448 n:127.0.0.1:63774_loks%2Fvq c:tmp_collection s:shard2 x:collection1 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:63774/loks/vq/collection1/ shard2
   [junit4]   2> 1000022 T6448 n:127.0.0.1:63774_loks%2Fvq c:tmp_collection s:shard2 x:collection1 oascc.SolrZkClient.makePath makePath: /collections/tmp_collection/leaders/shard2
   [junit4]   2> 1000022 T6440 n:127.0.0.1:63774_loks%2Fvq oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "operation":"leader",
   [junit4]   2> 	  "shard":"shard2",
   [junit4]   2> 	  "collection":"tmp_collection"} current state version: 1
   [junit4]   2> 1000024 T6439 n:127.0.0.1:63774_loks%2Fvq oash.CdcrLeaderStateManager$LeaderStateWatcher.process Received new leader state @ tmp_collection:shard2
   [junit4]   2> 1000027 T6449 n:127.0.0.1:63774_loks%2Fvq oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 1000028 T6440 n:127.0.0.1:63774_loks%2Fvq oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "operation":"leader",
   [junit4]   2> 	  "shard":"shard2",
   [junit4]   2> 	  "collection":"tmp_collection",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:63774/loks/vq",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "state":"active"} current state version: 1
   [junit4]   2> 1000110 T6449 n:127.0.0.1:63774_loks%2Fvq oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1000148 T6448 n:127.0.0.1:63774_loks%2Fvq c:tmp_collection s:shard2 x:collection1 oasc.ZkController.register We are http://127.0.0.1:63774/loks/vq/collection1/ and leader is http://127.0.0.1:63774/loks/vq/collection1/
   [junit4]   2> 1000148 T6448 n:127.0.0.1:63774_loks%2Fvq c:tmp_collection s:shard2 x:collection1 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:63774/loks/vq
   [junit4]   2> 1000148 T6448 n:127.0.0.1:63774_loks%2Fvq c:tmp_collection s:shard2 x:collection1 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
   [junit4]   2> 1000148 T6448 n:127.0.0.1:63774_loks%2Fvq c:tmp_collection s:shard2 x:collection1 oasc.ZkController.publish publishing core=collection1 state=active collection=tmp_collection
   [junit4]   2> 1000148 T6448 n:127.0.0.1:63774_loks%2Fvq c:tmp_collection s:shard2 x:collection1 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 1000149 T6449 n:127.0.0.1:63774_loks%2Fvq oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 1000151 T6440 n:127.0.0.1:63774_loks%2Fvq oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "core_node_name":"core_node1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"http://127.0.0.1:63774/loks/vq",
   [junit4]   2> 	  "node_name":"127.0.0.1:63774_loks%2Fvq",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "state":"active",
   [junit4]   2> 	  "shard":"shard2",
   [junit4]   2> 	  "collection":"tmp_collection",
   [junit4]   2> 	  "operation":"state"} current state version: 2
   [junit4]   2> 1000151 T6440 n:127.0.0.1:63774_loks%2Fvq oasco.ReplicaMutator.updateState Update state numShards=2 message={
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "core_node_name":"core_node1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"http://127.0.0.1:63774/loks/vq",
   [junit4]   2> 	  "node_name":"127.0.0.1:63774_loks%2Fvq",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "state":"active",
   [junit4]   2> 	  "shard":"shard2",
   [junit4]   2> 	  "collection":"tmp_collection",
   [junit4]   2> 	  "operation":"state"}
   [junit4]   2> 1000234 T6449 n:127.0.0.1:63774_loks%2Fvq oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1000245 T6411 oas.SolrTestCaseJ4.writeCoreProperties Writing core.properties file to C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CdcrRequestHandlerTest 2C54B4B78A95EA2F-001\jetty-002\cores\collection1
   [junit4]   2> 1000247 T6411 oejs.Server.doStart jetty-9.2.10.v20150310
   [junit4]   2> 1000249 T6411 oejsh.ContextHandler.doStart Started o.e.j.s.ServletContextHandler@140d4b8c{/loks/vq,null,AVAILABLE}
   [junit4]   2> 1000250 T6411 oejs.AbstractConnector.doStart Started ServerConnector@344e1eb4{HTTP/1.1}{127.0.0.1:63783}
   [junit4]   2> 1000250 T6411 oejs.Server.doStart Started @1001220ms
   [junit4]   2> 1000250 T6411 oascse.JettySolrRunner$1.lifeCycleStarted Jetty properties: {shards=shard2, hostContext=/loks/vq, hostPort=63783, coreRootDirectory=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CdcrRequestHandlerTest 2C54B4B78A95EA2F-001\jetty-002\cores}
   [junit4]   2> 1000251 T6411 oass.SolrDispatchFilter.init SolrDispatchFilter.init()sun.misc.Launcher$AppClassLoader@4e0e2f2a
   [junit4]   2> 1000251 T6411 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CdcrRequestHandlerTest 2C54B4B78A95EA2F-001\jetty-002\'
   [junit4]   2> 1000269 T6411 oasc.SolrXmlConfig.fromFile Loading container configuration from C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CdcrRequestHandlerTest 2C54B4B78A95EA2F-001\jetty-002\solr.xml
   [junit4]   2> 1000284 T6411 oasc.CorePropertiesLocator.<init> Config-defined core root directory: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CdcrRequestHandlerTest 2C54B4B78A95EA2F-001\jetty-002\cores
   [junit4]   2> 1000285 T6411 oasc.CoreContainer.<init> New CoreContainer 636255481
   [junit4]   2> 1000285 T6411 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CdcrRequestHandlerTest 2C54B4B78A95EA2F-001\jetty-002\]
   [junit4]   2> 1000285 T6411 oasc.CoreContainer.load loading shared library: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CdcrRequestHandlerTest 2C54B4B78A95EA2F-001\jetty-002\lib
   [junit4]   2> 1000285 T6411 oasc.SolrResourceLoader.addToClassLoader WARN Can't find (or read) directory to add to classloader: lib (resolved as: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CdcrRequestHandlerTest 2C54B4B78A95EA2F-001\jetty-002\lib).
   [junit4]   2> 1000292 T6411 oashc.HttpShardHandlerFactory.init created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1000295 T6411 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1000296 T6411 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1000296 T6411 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1000296 T6411 oasc.CoreContainer.load Node Name: 127.0.0.1
   [junit4]   2> 1000296 T6411 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:63767/solr
   [junit4]   2> 1000297 T6411 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 1000297 T6411 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 1000299 T6411 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1000300 T6463 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@75b9126d name:ZooKeeperConnection Watcher:127.0.0.1:63767 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1000302 T6411 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1000302 T6411 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 1000304 T6411 n:127.0.0.1:63783_loks%2Fvq oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1000306 T6466 n:127.0.0.1:63783_loks%2Fvq oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@318aefdc name:ZooKeeperConnection Watcher:127.0.0.1:63767/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1000306 T6411 n:127.0.0.1:63783_loks%2Fvq oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1000310 T6411 n:127.0.0.1:63783_loks%2Fvq oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 1001107 T6411 n:127.0.0.1:63783_loks%2Fvq oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:63783_loks%2Fvq
   [junit4]   2> 1001108 T6411 n:127.0.0.1:63783_loks%2Fvq oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:63783_loks%2Fvq
   [junit4]   2> 1001110 T6411 n:127.0.0.1:63783_loks%2Fvq oasc.Overseer.close Overseer (id=null) closing
   [junit4]   2> 1001112 T6411 n:127.0.0.1:63783_loks%2Fvq oasc.CoreContainer.initializeAuthenticationPlugin No authentication plugin used.
   [junit4]   2> 1001113 T6411 n:127.0.0.1:63783_loks%2Fvq oasc.CoreContainer.intializeAuthorizationPlugin Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1001113 T6411 n:127.0.0.1:63783_loks%2Fvq oasc.CorePropertiesLocator.discover Looking for core definitions underneath C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CdcrRequestHandlerTest 2C54B4B78A95EA2F-001\jetty-002\cores
   [junit4]   2> 1001114 T6411 n:127.0.0.1:63783_loks%2Fvq oasc.CoreDescriptor.<init> CORE DESCRIPTOR: {name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, loadOnStartup=true, instanceDir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CdcrRequestHandlerTest 2C54B4B78A95EA2F-001\jetty-002\cores\collection1, collection=tmp_collection, absoluteInstDir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CdcrRequestHandlerTest 2C54B4B78A95EA2F-001\jetty-002\cores\collection1\, coreNodeName=, dataDir=data\, shard=}
   [junit4]   2> 1001114 T6411 n:127.0.0.1:63783_loks%2Fvq oasc.CorePropertiesLocator.discoverUnder Found core collection1 in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CdcrRequestHandlerTest 2C54B4B78A95EA2F-001\jetty-002\cores\collection1\
   [junit4]   2> 1001114 T6411 n:127.0.0.1:63783_loks%2Fvq oasc.CorePropertiesLocator.discover Found 1 core definitions
   [junit4]   2> 1001115 T6467 n:127.0.0.1:63783_loks%2Fvq c:tmp_collection x:collection1 oasc.ZkController.publish publishing core=collection1 state=down collection=tmp_collection
   [junit4]   2> 1001115 T6467 n:127.0.0.1:63783_loks%2Fvq c:tmp_collection x:collection1 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 1001116 T6449 n:127.0.0.1:63774_loks%2Fvq oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 1001116 T6467 n:127.0.0.1:63783_loks%2Fvq oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 1001117 T6440 n:127.0.0.1:63774_loks%2Fvq oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"http://127.0.0.1:63783/loks/vq",
   [junit4]   2> 	  "node_name":"127.0.0.1:63783_loks%2Fvq",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "collection":"tmp_collection",
   [junit4]   2> 	  "operation":"state"} current state version: 3
   [junit4]   2> 1001117 T6440 n:127.0.0.1:63774_loks%2Fvq oasco.ReplicaMutator.updateState Update state numShards=2 message={
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"http://127.0.0.1:63783/loks/vq",
   [junit4]   2> 	  "node_name":"127.0.0.1:63783_loks%2Fvq",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "collection":"tmp_collection",
   [junit4]   2> 	  "operation":"state"}
   [junit4]   2> 1001117 T6440 n:127.0.0.1:63774_loks%2Fvq oasco.ReplicaMutator.updateState Collection already exists with numShards=2
   [junit4]   2> 1001117 T6440 n:127.0.0.1:63774_loks%2Fvq oasco.ReplicaMutator.updateState Assigning new node to shard shard=shard1
   [junit4]   2> 1001199 T6466 n:127.0.0.1:63783_loks%2Fvq oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1001199 T6449 n:127.0.0.1:63774_loks%2Fvq oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1001910 T6467 n:127.0.0.1:63783_loks%2Fvq oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1001910 T6467 n:127.0.0.1:63783_loks%2Fvq oasc.ZkController.createCollectionZkNode Check for collection zkNode:tmp_collection
   [junit4]   2> 1001911 T6467 n:127.0.0.1:63783_loks%2Fvq oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 1001911 T6467 n:127.0.0.1:63783_loks%2Fvq oascc.ZkStateReader.readConfigName Load collection config from:/collections/tmp_collection
   [junit4]   2> 1001911 T6467 n:127.0.0.1:63783_loks%2Fvq oascc.ZkStateReader.readConfigName path=/collections/tmp_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1001911 T6467 n:127.0.0.1:63783_loks%2Fvq oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CdcrRequestHandlerTest 2C54B4B78A95EA2F-001\jetty-002\cores\collection1\'
   [junit4]   2> 1001921 T6467 n:127.0.0.1:63783_loks%2Fvq oasc.Config.<init> loaded config solrconfig.xml with version 0 
   [junit4]   2> 1001926 T6467 n:127.0.0.1:63783_loks%2Fvq oasc.SolrConfig.refreshRequestParams current version of requestparams : -1
   [junit4]   2> 1001928 T6467 n:127.0.0.1:63783_loks%2Fvq oasc.SolrConfig.<init> Using Lucene MatchVersion: 6.0.0
   [junit4]   2> 1001932 T6467 n:127.0.0.1:63783_loks%2Fvq oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1001933 T6467 n:127.0.0.1:63783_loks%2Fvq oass.IndexSchema.readSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 1001936 T6467 n:127.0.0.1:63783_loks%2Fvq oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 1001999 T6467 n:127.0.0.1:63783_loks%2Fvq oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 1002000 T6467 n:127.0.0.1:63783_loks%2Fvq oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 1002000 T6467 n:127.0.0.1:63783_loks%2Fvq oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 1002002 T6467 n:127.0.0.1:63783_loks%2Fvq oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 1002017 T6467 n:127.0.0.1:63783_loks%2Fvq oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection tmp_collection
   [junit4]   2> 1002017 T6467 n:127.0.0.1:63783_loks%2Fvq x:collection1 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 1002018 T6467 n:127.0.0.1:63783_loks%2Fvq x:collection1 oasc.SolrCore.<init> [[collection1] ] Opening new SolrCore at [C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CdcrRequestHandlerTest 2C54B4B78A95EA2F-001\jetty-002\cores\collection1\], dataDir=[null]
   [junit4]   2> 1002018 T6467 n:127.0.0.1:63783_loks%2Fvq x:collection1 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@573d2c90
   [junit4]   2> 1002018 T6467 n:127.0.0.1:63783_loks%2Fvq x:collection1 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CdcrRequestHandlerTest 2C54B4B78A95EA2F-001\jetty-002\cores\collection1\data\
   [junit4]   2> 1002018 T6467 n:127.0.0.1:63783_loks%2Fvq x:collection1 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CdcrRequestHandlerTest 2C54B4B78A95EA2F-001\jetty-002\cores\collection1\data\index/
   [junit4]   2> 1002018 T6467 n:127.0.0.1:63783_loks%2Fvq x:collection1 oasc.SolrCore.initIndex WARN [collection1] Solr index directory 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CdcrRequestHandlerTest 2C54B4B78A95EA2F-001\jetty-002\cores\collection1\data\index' doesn't exist. Creating new index...
   [junit4]   2> 1002018 T6467 n:127.0.0.1:63783_loks%2Fvq x:collection1 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CdcrRequestHandlerTest 2C54B4B78A95EA2F-001\jetty-002\cores\collection1\data\index
   [junit4]   2> 1002018 T6467 n:127.0.0.1:63783_loks%2Fvq x:collection1 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=35, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.8468594779792245]
   [junit4]   2> 1002019 T6467 n:127.0.0.1:63783_loks%2Fvq x:collection1 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@2d06cc3f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@13c22d7e),segFN=segments_1,generation=1}
   [junit4]   2> 1002019 T6467 n:127.0.0.1:63783_loks%2Fvq x:collection1 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 1002021 T6467 n:127.0.0.1:63783_loks%2Fvq x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "cdcr-processor-chain"
   [junit4]   2> 1002021 T6467 n:127.0.0.1:63783_loks%2Fvq x:collection1 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 1002024 T6467 n:127.0.0.1:63783_loks%2Fvq x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 1002025 T6467 n:127.0.0.1:63783_loks%2Fvq x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 1002025 T6467 n:127.0.0.1:63783_loks%2Fvq x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 1002026 T6467 n:127.0.0.1:63783_loks%2Fvq x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 1002026 T6467 n:127.0.0.1:63783_loks%2Fvq x:collection1 oasc.PluginBag.createPlugin requestHandler : '/replication' created with startup=lazy 
   [junit4]   2> 1002028 T6467 n:127.0.0.1:63783_loks%2Fvq x:collection1 oasc.SolrResourceLoader.findClass WARN Solr loaded a deprecated plugin/analysis class [org.apache.solr.handler.admin.AdminHandlers]. Please consult documentation how to replace it accordingly.
   [junit4]   2> 1002028 T6467 n:127.0.0.1:63783_loks%2Fvq x:collection1 oasc.SolrResourceLoader.findClass WARN Solr loaded a deprecated plugin/analysis class [org.apache.solr.handler.admin.AdminHandlers]. Please consult documentation how to replace it accordingly.
   [junit4]   2> 1002028 T6467 n:127.0.0.1:63783_loks%2Fvq x:collection1 oasc.RequestHandlers.initHandlersFromConfig Registered paths: /admin/mbeans,standard,/update/csv,/update/json/docs,/admin/luke,/admin/segments,/get,/admin/system,/replication,/admin/properties,/config,/schema,/admin/plugins,/admin/logging,/update/json,/admin,/admin/threads,/admin/ping,/cdcr,/update,/admin/file
   [junit4]   2> 1002029 T6467 n:127.0.0.1:63783_loks%2Fvq x:collection1 oasc.SolrCore.initStatsCache Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 1002029 T6467 n:127.0.0.1:63783_loks%2Fvq x:collection1 oasu.UpdateHandler.<init> Using UpdateLog implementation: org.apache.solr.update.CdcrUpdateLog
   [junit4]   2> 1002030 T6467 n:127.0.0.1:63783_loks%2Fvq x:collection1 oasu.UpdateLog.init Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1002039 T6467 n:127.0.0.1:63783_loks%2Fvq x:collection1 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 1002039 T6467 n:127.0.0.1:63783_loks%2Fvq x:collection1 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 1002040 T6467 n:127.0.0.1:63783_loks%2Fvq x:collection1 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=34, maxMergeAtOnceExplicit=20, maxMergedSegmentMB=44.330078125, floorSegmentMB=0.318359375, forceMergeDeletesPctAllowed=25.688269165098806, segmentsPerTier=21.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
   [junit4]   2> 1002042 T6467 n:127.0.0.1:63783_loks%2Fvq x:collection1 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@2d06cc3f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@13c22d7e),segFN=segments_1,generation=1}
   [junit4]   2> 1002042 T6467 n:127.0.0.1:63783_loks%2Fvq x:collection1 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 1002042 T6467 n:127.0.0.1:63783_loks%2Fvq x:collection1 oass.SolrIndexSearcher.<init> Opening Searcher@60f71dfb[collection1] main
   [junit4]   2> 1002042 T6467 n:127.0.0.1:63783_loks%2Fvq x:collection1 oasu.UpdateLog.onFirstSearcher On first searcher opened, looking up max value of version field
   [junit4]   2> 1002042 T6467 n:127.0.0.1:63783_loks%2Fvq x:collection1 oasu.VersionInfo.getMaxVersionFromIndex Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 1002042 T6467 n:127.0.0.1:63783_loks%2Fvq x:collection1 oasu.VersionInfo.getMaxVersionFromIndex WARN No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 1002042 T6467 n:127.0.0.1:63783_loks%2Fvq x:collection1 oasu.UpdateLog.seedBucketsWithHighestVersion WARN Could not find max version in index or recent updates, using new clock 1501908556390072320
   [junit4]   2> 1002044 T6467 n:127.0.0.1:63783_loks%2Fvq x:collection1 oasu.UpdateLog.seedBucketsWithHighestVersion Took 2 ms to seed version buckets with highest version 1501908556390072320
   [junit4]   2> 1002044 T6467 n:127.0.0.1:63783_loks%2Fvq x:collection1 oascc.ZkStateReader.readConfigName Load collection config from:/collections/tmp_collection
   [junit4]   2> 1002045 T6467 n:127.0.0.1:63783_loks%2Fvq x:collection1 oascc.ZkStateReader.readConfigName path=/collections/tmp_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1002045 T6467 n:127.0.0.1:63783_loks%2Fvq x:collection1 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 1002046 T6467 n:127.0.0.1:63783_loks%2Fvq x:collection1 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 1002046 T6467 n:127.0.0.1:63783_loks%2Fvq x:collection1 oasr.RestManager.init Initializing RestManager with initArgs: {}
   [junit4]   2> 1002046 T6467 n:127.0.0.1:63783_loks%2Fvq x:collection1 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1002046 T6467 n:127.0.0.1:63783_loks%2Fvq x:collection1 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 1002047 T6467 n:127.0.0.1:63783_loks%2Fvq x:collection1 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1002047 T6467 n:127.0.0.1:63783_loks%2Fvq x:collection1 oasr.RestManager.init Initializing 0 registered ManagedResources
   [junit4]   2> 1002049 T6467 n:127.0.0.1:63783_loks%2Fvq x:collection1 oasha.AdminHandlers.inform WARN <requestHandler name="/admin/" 
   [junit4]   2> 	 class="solr.admin.AdminHandlers" /> is deprecated . It is not required anymore
   [junit4]   2> 1002051 T6468 n:127.0.0.1:63783_loks%2Fvq x:collection1 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@60f71dfb[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1002051 T6467 n:127.0.0.1:63783_loks%2Fvq x:collection1 oasc.ZkController.getConfDirListeners watch zkdir /configs/conf1
   [junit4]   2> 1002052 T6467 n:127.0.0.1:63783_loks%2Fvq x:collection1 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 1002052 T6472 n:127.0.0.1:63783_loks%2Fvq c:tmp_collection s:shard1 x:collection1 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:63783/loks/vq collection:tmp_collection shard:shard1
   [junit4]   2> 1002052 T6411 n:127.0.0.1:63783_loks%2Fvq oass.SolrDispatchFilter.init user.dir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J1
   [junit4]   2> 1002053 T6411 n:127.0.0.1:63783_loks%2Fvq oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 1002054 T6472 n:127.0.0.1:63783_loks%2Fvq c:tmp_collection s:shard1 x:collection1 oascc.SolrZkClient.makePath makePath: /collections/tmp_collection/leader_elect/shard1/election
   [junit4]   2> 1002059 T6472 n:127.0.0.1:63783_loks%2Fvq c:tmp_collection s:shard1 x:collection1 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
   [junit4]   2> 1002061 T6449 n:127.0.0.1:63774_loks%2Fvq oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 1002062 T6472 n:127.0.0.1:63783_loks%2Fvq c:tmp_collection s:shard1 x:collection1 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
   [junit4]   2> 1002062 T6472 n:127.0.0.1:63783_loks%2Fvq c:tmp_collection s:shard1 x:collection1 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
   [junit4]   2> ASYNC  NEW_CORE C2885 name=collection1 org.apache.solr.core.SolrCore@358e2f06 url=http://127.0.0.1:63783/loks/vq/collection1 node=127.0.0.1:63783_loks%2Fvq C2885_STATE=coll:tmp_collection core:collection1 props:{core=collection1, base_url=http://127.0.0.1:63783/loks/vq, node_name=127.0.0.1:63783_loks%2Fvq, state=down}
   [junit4]   2> 1002062 T6472 n:127.0.0.1:63783_loks%2Fvq c:tmp_collection s:shard1 x:collection1 C2885 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:63783/loks/vq/collection1/
   [junit4]   2> 1002062 T6472 n:127.0.0.1:63783_loks%2Fvq c:tmp_collection s:shard1 x:collection1 C2885 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
   [junit4]   2> 1002063 T6472 n:127.0.0.1:63783_loks%2Fvq c:tmp_collection s:shard1 x:collection1 C2885 oasc.SyncStrategy.syncToMe http://127.0.0.1:63783/loks/vq/collection1/ has no replicas
   [junit4]   2> 1002063 T6472 n:127.0.0.1:63783_loks%2Fvq c:tmp_collection s:shard1 x:collection1 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:63783/loks/vq/collection1/ shard1
   [junit4]   2> 1002064 T6472 n:127.0.0.1:63783_loks%2Fvq c:tmp_collection s:shard1 x:collection1 oascc.SolrZkClient.makePath makePath: /collections/tmp_collection/leaders/shard1
   [junit4]   2> 1002066 T6440 n:127.0.0.1:63774_loks%2Fvq oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "operation":"leader",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "collection":"tmp_collection"} current state version: 4
   [junit4]   2> 1002069 T6466 n:127.0.0.1:63783_loks%2Fvq oash.CdcrLeaderStateManager$LeaderStateWatcher.process Received new leader state @ tmp_collection:shard1
   [junit4]   2> 1002073 T6449 n:127.0.0.1:63774_loks%2Fvq oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 1002076 T6440 n:127.0.0.1:63774_loks%2Fvq oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "operation":"leader",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "collection":"tmp_collection",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:63783/loks/vq",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "state":"active"} current state version: 4
   [junit4]   2> 1002162 T6466 n:127.0.0.1:63783_loks%2Fvq oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1002163 T6449 n:127.0.0.1:63774_loks%2Fvq oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1002197 T6472 n:127.0.0.1:63783_loks%2Fvq c:tmp_collection s:shard1 x:collection1 oasc.ZkController.register We are http://127.0.0.1:63783/loks/vq/collection1/ and leader is http://127.0.0.1:63783/loks/vq/collection1/
   [junit4]   2> 1002197 T6472 n:127.0.0.1:63783_loks%2Fvq c:tmp_collection s:shard1 x:collection1 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:63783/loks/vq
   [junit4]   2> 1002197 T6472 n:127.0.0.1:63783_loks%2Fvq c:tmp_collection s:shard1 x:collection1 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
   [junit4]   2> 1002197 T6472 n:127.0.0.1:63783_loks%2Fvq c:tmp_collection s:shard1 x:collection1 oasc.ZkController.publish publishing core=collection1 state=active collection=tmp_collection
   [junit4]   2> 1002197 T6472 n:127.0.0.1:63783_loks%2Fvq c:tmp_collection s:shard1 x:collection1 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 1002201 T6449 n:127.0.0.1:63774_loks%2Fvq oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 1002203 T6440 n:127.0.0.1:63774_loks%2Fvq oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "core_node_name":"core_node2",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"http://127.0.0.1:63783/loks/vq",
   [junit4]   2> 	  "node_name":"127.0.0.1:63783_loks%2Fvq",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "state":"active",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "collection":"tmp_collection",
   [junit4]   2> 	  "operation":"state"} current state version: 5
   [junit4]   2> 1002204 T6440 n:127.0.0.1:63774_loks%2Fvq oasco.ReplicaMutator.updateState Update state numShards=2 message={
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "core_node_name":"core_node2",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"http://127.0.0.1:63783/loks/vq",
   [junit4]   2> 	  "node_name":"127.0.0.1:63783_loks%2Fvq",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "state":"active",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "collection":"tmp_collection",
   [junit4]   2> 	  "operation":"state"}
   [junit4]   2> 1002287 T6466 n:127.0.0.1:63783_loks%2Fvq oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1002287 T6449 n:127.0.0.1:63774_loks%2Fvq oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1002389 T6411 oas.SolrTestCaseJ4.writeCoreProperties Writing core.properties file to C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CdcrRequestHandlerTest 2C54B4B78A95EA2F-001\jetty-003\cores\collection1
   [junit4]   2> 1002391 T6411 oejs.Server.doStart jetty-9.2.10.v20150310
   [junit4]   2> 1002393 T6411 oejsh.ContextHandler.doStart Started o.e.j.s.ServletContextHandler@6c84ba2e{/loks/vq,null,AVAILABLE}
   [junit4]   2> 1002394 T6411 oejs.AbstractConnector.doStart Started ServerConnector@5263be1{HTTP/1.1}{127.0.0.1:63792}
   [junit4]   2> 1002394 T6411 oejs.Server.doStart Started @1003930ms
   [junit4]   2> 1002394 T6411 oascse.JettySolrRunner$1.lifeCycleStarted Jetty properties: {shards=shard3, hostContext=/loks/vq, hostPort=63792, coreRootDirectory=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CdcrRequestHandlerTest 2C54B4B78A95EA2F-001\jetty-003\cores}
   [junit4]   2> 1002394 T6411 oass.SolrDispatchFilter.init SolrDispatchFilter.init()sun.misc.Launcher$AppClassLoader@4e0e2f2a
   [junit4]   2> 1002394 T6411 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CdcrRequestHandlerTest 2C54B4B78A95EA2F-001\jetty-003\'
   [junit4]   2> 1002412 T6411 oasc.SolrXmlConfig.fromFile Loading container configuration from C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CdcrRequestHandlerTest 2C54B4B78A95EA2F-001\jetty-003\solr.xml
   [junit4]   2> 1002418 T6411 oasc.CorePropertiesLocator.<init> Config-defined core root directory: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CdcrRequestHandlerTest 2C54B4B78A95EA2F-001\jetty-003\cores
   [junit4]   2> 1002418 T6411 oasc.CoreContainer.<init> New CoreContainer 1582047771
   [junit4]   2> 1002418 T6411 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CdcrRequestHandlerTest 2C54B4B78A95EA2F-001\jetty-003\]
   [junit4]   2> 1002418 T6411 oasc.CoreContainer.load loading shared library: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CdcrRequestHandlerTest 2C54B4B78A95EA2F-001\jetty-003\lib
   [junit4]   2> 1002418 T6411 oasc.SolrResourceLoader.addToClassLoader WARN Can't find (or read) directory to add to classloader: lib (resolved as: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CdcrRequestHandlerTest 2C54B4B78A95EA2F-001\jetty-003\lib).
   [junit4]   2> 1002425 T6411 oashc.HttpShardHandlerFactory.init created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1002428 T6411 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1002428 T6411 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1002428 T6411 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1002428 T6411 oasc.CoreContainer.load Node Name: 127.0.0.1
   [junit4]   2> 1002428 T6411 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:63767/solr
   [junit4]   2> 1002429 T6411 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 1002429 T6411 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 1002430 T6411 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1002432 T6486 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@273c3f3e name:ZooKeeperConnection Watcher:127.0.0.1:63767 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1002432 T6411 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1002432 T6411 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 1002434 T6413 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x14d7d9deffc0006, likely client has closed socket
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 
   [junit4]   2> 1002435 T6411 n:127.0.0.1:63792_loks%2Fvq oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1002436 T6489 n:127.0.0.1:63792_loks%2Fvq oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@55c5a96d name:ZooKeeperConnection Watcher:127.0.0.1:63767/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1002436 T6411 n:127.0.0.1:63792_loks%2Fvq oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1002441 T6411 n:127.0.0.1:63792_loks%2Fvq oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 1003239 T6411 n:127.0.0.1:63792_loks%2Fvq oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:63792_loks%2Fvq
   [junit4]   2> 1003240 T6411 n:127.0.0.1:63792_loks%2Fvq oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:63792_loks%2Fvq
   [junit4]   2> 1003244 T6411 n:127.0.0.1:63792_loks%2Fvq oasc.Overseer.close Overseer (id=null) closing
   [junit4]   2> 1003246 T6411 n:127.0.0.1:63792_loks%2Fvq oasc.CoreContainer.initializeAuthenticationPlugin No authentication plugin used.
   [junit4]   2> 

[...truncated too long message...]

at org.apache.solr.util.BaseTestHarness.validateUpdate(BaseTestHarness.java:252)
   [junit4]   2> 	        at org.apache.solr.SolrTestCaseJ4.checkUpdateU(SolrTestCaseJ4.java:748)
   [junit4]   2> 	        at org.apache.solr.SolrTestCaseJ4.assertU(SolrTestCaseJ4.java:727)
   [junit4]   2> 	        at org.apache.solr.SolrTestCaseJ4.assertU(SolrTestCaseJ4.java:721)
   [junit4]   2> 	        at org.apache.solr.update.processor.AtomicUpdatesTest.before(AtomicUpdatesTest.java:45)
   [junit4]   2> 	        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   2> 	        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   [junit4]   2> 	        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2> 	        at java.lang.reflect.Method.invoke(Method.java:497)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:870)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2> 	        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
   [junit4]   2> 	        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2> 	        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
   [junit4]   2> 	        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2> 	        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2> 	        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   2> 	        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2> 	        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2> 	        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   2> 	        at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 	   3) Thread[id=3916, name=searcherExecutor-1756-thread-1, state=WAITING, group=TGRP-AtomicUpdatesTest]
   [junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
   [junit4]   2> 	        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
   [junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
   [junit4]   2> 	        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
   [junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]   2> 	        at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> NOTE: test params are: codec=DummyCompressingStoredFields(storedFieldsFormat=CompressingStoredFieldsFormat(compressionMode=DUMMY, chunkSize=3, maxDocsPerChunk=4, blockSize=4), termVectorsFormat=CompressingTermVectorsFormat(compressionMode=DUMMY, chunkSize=3, blockSize=4)), sim=DefaultSimilarity, locale=es_BO, timezone=Indian/Mayotte
   [junit4]   2> NOTE: Windows 7 6.1 amd64/Oracle Corporation 1.8.0_45 (64-bit)/cpus=3,threads=4,free=176500440,total=377323520
   [junit4]   2> NOTE: All tests run in this JVM: [TestCursorMarkWithoutUniqueKey, ResponseLogComponentTest, TestExpandComponent, TestQueryTypes, DocValuesMultiTest, TestPerFieldSimilarity, CoreAdminRequestStatusTest, TestConfigSets, TestSolrQueryParserDefaultOperatorResource, RequiredFieldsTest, TestDynamicFieldResource, TestShardHandlerFactory, TestCollationFieldDocValues, TestSchemaManager, TestHighlightDedupGrouping, TestCryptoKeys, DistributedSuggestComponentTest, DirectSolrConnectionTest, AlternateDirectoryTest, SolrCoreCheckLockOnStartupTest, CoreAdminHandlerTest, TestHashQParserPlugin, TestBinaryResponseWriter, TestCloudSchemaless, SoftAutoCommitTest, LeaderFailoverAfterPartitionTest, DocExpirationUpdateProcessorFactoryTest, FieldAnalysisRequestHandlerTest, RulesTest, TestFieldTypeCollectionResource, PingRequestHandlerTest, TestManagedSynonymFilterFactory, ShardRoutingTest, SolrPluginUtilsTest, EnumFieldTest, TestImplicitCoreProperties, TestComplexPhraseQParserPlugin, ScriptEngineTest, SolrIndexSplitterTest, OverseerTest, TestHashPartitioner, TestFieldCollectionResource, DeleteInactiveReplicaTest, TestFieldResource, TestCoreContainer, TestReRankQParserPlugin, SimpleFacetsTest, DistributedSpellCheckComponentTest, TestExtendedDismaxParser, SuggesterFSTTest, TestInitQParser, HdfsSyncSliceTest, SyncSliceTest, OverriddenZkACLAndCredentialsProvidersTest, TestDocumentBuilder, SampleTest, SolrInfoMBeanTest, SOLR749Test, TestReplicaProperties, AtomicUpdatesTest]
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=AtomicUpdatesTest -Dtests.seed=2C54B4B78A95EA2F -Dtests.slow=true -Dtests.locale=es_BO -Dtests.timezone=Indian/Mayotte -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR   0.00s J0 | AtomicUpdatesTest (suite) <<<
   [junit4]    > Throwable #1: java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([2C54B4B78A95EA2F]:0)
   [junit4] Completed [439/502] on J0 in 7288.34s, 1 test, 2 errors <<< FAILURES!

[...truncated 194 lines...]
BUILD FAILED
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\build.xml:526: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\build.xml:474: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\build.xml:61: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\extra-targets.xml:39: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build.xml:229: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\common-build.xml:512: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\common-build.xml:1415: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\common-build.xml:973: There were test failures: 502 suites, 1971 tests, 1 suite-level error, 1 error, 3 failures, 226 ignored (32 assumptions)

Total time: 151 minutes 15 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-trunk-Windows (64bit/jdk1.8.0_45) - Build # 4841 - Still Failing!

Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Windows/4841/
Java: 64bit/jdk1.8.0_45 -XX:+UseCompressedOops -XX:+UseSerialGC

2 tests failed.
FAILED:  org.apache.solr.CursorPagingTest.testBadInputs

Error Message:
Test abandoned because suite timeout was reached.

Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
	at __randomizedtesting.SeedInfo.seed([1A319452CA93DFC9]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.CursorPagingTest

Error Message:
Suite timeout exceeded (>= 7200000 msec).

Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
	at __randomizedtesting.SeedInfo.seed([1A319452CA93DFC9]:0)




Build Log:
[...truncated 10687 lines...]
   [junit4] Suite: org.apache.solr.CursorPagingTest
   [junit4]   2> Creating dataDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J1\temp\solr.CursorPagingTest 1A319452CA93DFC9-001\init-core-data-001
   [junit4]   2> 614303 T4129 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (false) and clientAuth (false)
   [junit4]   2> 614305 T4129 oas.SolrTestCaseJ4.initCore ####initCore
   [junit4]   2> 614305 T4129 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\'
   [junit4]   2> 614306 T4129 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/C:/Users/JenkinsSlave/workspace/Lucene-Solr-trunk-Windows/solr/core/src/test-files/solr/collection1/lib/classes/' to classloader
   [junit4]   2> 614306 T4129 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/C:/Users/JenkinsSlave/workspace/Lucene-Solr-trunk-Windows/solr/core/src/test-files/solr/collection1/lib/README' to classloader
   [junit4]   2> 614348 T4129 oasc.SolrConfig.refreshRequestParams current version of requestparams : -1
   [junit4]   2> 614351 T4129 oasc.SolrConfig.<init> Using Lucene MatchVersion: 6.0.0
   [junit4]   2> 614383 T4129 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig-deeppaging.xml
   [junit4]   2> 614383 T4129 oass.IndexSchema.readSchema Reading Solr Schema from C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\schema-sorts.xml
   [junit4]   2> 614386 T4129 oass.IndexSchema.readSchema [null] Schema name=sortable
   [junit4]   2> 614918 T4129 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 614919 T4129 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 614919 T4129 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr
   [junit4]   2> 614919 T4129 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\'
   [junit4]   2> 614941 T4129 oasc.CoreContainer.<init> New CoreContainer 495221275
   [junit4]   2> 614941 T4129 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\]
   [junit4]   2> 614941 T4129 oasc.CoreContainer.load loading shared library: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\lib
   [junit4]   2> 614941 T4129 oasc.SolrResourceLoader.addToClassLoader WARN Can't find (or read) directory to add to classloader: lib (resolved as: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\lib).
   [junit4]   2> 614950 T4129 oashc.HttpShardHandlerFactory.init created with socketTimeout : 600000,connTimeout : 60000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 614953 T4129 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=30000&connTimeout=30000&retry=true
   [junit4]   2> 614954 T4129 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 614954 T4129 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 614954 T4129 oasc.CoreContainer.load Node Name: testNode
   [junit4]   2> 614954 T4129 oasc.CoreContainer.initializeAuthenticationPlugin No authentication plugin used.
   [junit4]   2> 614955 T4129 oasc.CoreDescriptor.<init> CORE DESCRIPTOR: {name=collection1, config=solrconfig-deeppaging.xml, transient=false, schema=schema-sorts.xml, loadOnStartup=true, instanceDir=collection1, collection=collection1, absoluteInstDir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\, dataDir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J1\temp\solr.CursorPagingTest 1A319452CA93DFC9-001\init-core-data-001, shard=shard1}
   [junit4]   2> 614956 T4130 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\'
   [junit4]   2> 614957 T4130 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/C:/Users/JenkinsSlave/workspace/Lucene-Solr-trunk-Windows/solr/core/src/test-files/solr/collection1/lib/classes/' to classloader
   [junit4]   2> 614957 T4130 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/C:/Users/JenkinsSlave/workspace/Lucene-Solr-trunk-Windows/solr/core/src/test-files/solr/collection1/lib/README' to classloader
   [junit4]   2> 614982 T4130 oasc.SolrConfig.refreshRequestParams current version of requestparams : -1
   [junit4]   2> 614992 T4130 oasc.SolrConfig.<init> Using Lucene MatchVersion: 6.0.0
   [junit4]   2> 614994 T4130 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig-deeppaging.xml
   [junit4]   2> 614995 T4130 oass.IndexSchema.readSchema Reading Solr Schema from C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\schema-sorts.xml
   [junit4]   2> 614998 T4130 oass.IndexSchema.readSchema [collection1] Schema name=sortable
   [junit4]   2> 615250 T4130 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 615250 T4130 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from instancedir C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\
   [junit4]   2> 615251 T4130 x:collection1 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 615258 T4130 x:collection1 oasc.SolrCore.<init> [[collection1] ] Opening new SolrCore at [C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\], dataDir=[null]
   [junit4]   2> 615258 T4130 x:collection1 oasc.SolrCore.initInfoRegistry JMX monitoring not detected for core: collection1
   [junit4]   2> 615258 T4130 x:collection1 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J1\temp\solr.CursorPagingTest 1A319452CA93DFC9-001\init-core-data-001\
   [junit4]   2> 615258 T4130 x:collection1 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J1\temp\solr.CursorPagingTest 1A319452CA93DFC9-001\init-core-data-001\index/
   [junit4]   2> 615259 T4130 x:collection1 oasc.SolrCore.initIndex WARN [collection1] Solr index directory 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J1\temp\solr.CursorPagingTest 1A319452CA93DFC9-001\init-core-data-001\index' doesn't exist. Creating new index...
   [junit4]   2> 615259 T4130 x:collection1 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J1\temp\solr.CursorPagingTest 1A319452CA93DFC9-001\init-core-data-001\index
   [junit4]   2> 615259 T4130 x:collection1 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=45, maxMergeAtOnceExplicit=38, maxMergedSegmentMB=23.3154296875, floorSegmentMB=1.1953125, forceMergeDeletesPctAllowed=21.993142861363214, segmentsPerTier=14.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.2698013129079748
   [junit4]   2> 615260 T4130 x:collection1 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@57d438b8 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@494349b9),segFN=segments_1,generation=1}
   [junit4]   2> 615260 T4130 x:collection1 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 615262 T4130 x:collection1 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 615265 T4130 x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 615266 T4130 x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 615267 T4130 x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 615268 T4130 x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 615269 T4130 x:collection1 oasc.RequestHandlers.initHandlersFromConfig Registered paths: /admin/mbeans,,/update/csv,/update/json/docs,/admin/luke,/select,/admin/segments,/get,/admin/system,/replication,/admin/properties,/config,/schema,/admin/plugins,/admin/logging,/update/json,/admin/threads,/admin/ping,/update,/admin/file
   [junit4]   2> 615270 T4130 x:collection1 oasc.RequestHandlers.initHandlersFromConfig WARN no default request handler is registered (either '/select' or 'standard')
   [junit4]   2> 615270 T4130 x:collection1 oasc.SolrCore.initStatsCache Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 615271 T4130 x:collection1 oasu.UpdateHandler.<init> Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 615271 T4130 x:collection1 oasu.UpdateLog.init Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=256
   [junit4]   2> 615272 T4130 x:collection1 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 615272 T4130 x:collection1 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 615272 T4130 x:collection1 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=18, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.21718263847112818]
   [junit4]   2> 615272 T4130 x:collection1 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@57d438b8 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@494349b9),segFN=segments_1,generation=1}
   [junit4]   2> 615272 T4130 x:collection1 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 615273 T4130 x:collection1 oass.SolrIndexSearcher.<init> Opening Searcher@2efad93c[collection1] main
   [junit4]   2> 615273 T4130 x:collection1 oasu.UpdateLog.onFirstSearcher On first searcher opened, looking up max value of version field
   [junit4]   2> 615273 T4130 x:collection1 oasu.VersionInfo.getMaxVersionFromIndex Refreshing highest value of _version_ for 256 version buckets from index
   [junit4]   2> 615273 T4130 x:collection1 oasu.VersionInfo.getMaxVersionFromIndex WARN No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 615273 T4130 x:collection1 oasu.UpdateLog.seedBucketsWithHighestVersion WARN Could not find max version in index or recent updates, using new clock 1501871705659801600
   [junit4]   2> 615273 T4130 x:collection1 oasu.UpdateLog.seedBucketsWithHighestVersion Took 0 ms to seed version buckets with highest version 1501871705659801600
   [junit4]   2> 615274 T4130 x:collection1 oasr.ManagedResourceStorage.newStorageIO WARN Cannot write to config directory C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf; switching to use InMemory storage instead.
   [junit4]   2> 615274 T4130 x:collection1 oasr.RestManager.init Initializing RestManager with initArgs: {}
   [junit4]   2> 615274 T4130 x:collection1 oasr.ManagedResourceStorage.load Reading _rest_managed.json using InMemoryStorage
   [junit4]   2> 615275 T4130 x:collection1 oasr.RestManager.init Initializing 0 registered ManagedResources
   [junit4]   2> 615275 T4130 x:collection1 oash.ReplicationHandler.inform Commits will be reserved for  10000
   [junit4]   2> 615275 T4130 x:collection1 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 615276 T4129 oas.SolrTestCaseJ4.initCore ####initCore end
   [junit4]   2> 615361 T4129 oas.SolrTestCaseJ4.setUp ###Starting testBadInputs
   [junit4]   2> ASYNC  NEW_CORE C3547 name=collection1 org.apache.solr.core.SolrCore@4697f9b5
   [junit4]   2> 615384 T4129 C3547 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[42 (1501871705752076288)]} 0 23
   [junit4]   2> 615387 T4129 C3547 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[66 (1501871705778290688)]} 0 1
   [junit4]   2> 615387 T4129 C3547 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 615416 T4129 C3547 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@57d438b8 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@494349b9),segFN=segments_1,generation=1}
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@57d438b8 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@494349b9),segFN=segments_2,generation=2}
   [junit4]   2> 615416 T4129 C3547 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
   [junit4]   2> 615417 T4129 C3547 oasc.SolrCore.getSearcher WARN [collection1] PERFORMANCE WARNING: Overlapping onDeckSearchers=2
   [junit4]   2> 615421 T4129 C3547 oass.SolrIndexSearcher.<init> Opening Searcher@25d8c9a7[collection1] main
   [junit4]   2> 615421 T4129 C3547 oasu.UpdateLog.onFirstSearcher On first searcher opened, looking up max value of version field
   [junit4]   2> 615422 T4131 x:collection1 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2efad93c[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 7214227 T4128 ccr.ThreadLeakControl$2.evaluate WARNING Suite execution timed out: org.apache.solr.CursorPagingTest
   [junit4]   2> 	==== jstack at approximately timeout time ====
   [junit4]   2> 	"Thread-838" ID=4132 WAITING on java.lang.Object@30733378
   [junit4]   2> 		at java.lang.Object.wait(Native Method)
   [junit4]   2> 		- waiting on java.lang.Object@30733378
   [junit4]   2> 		at java.lang.Object.wait(Object.java:502)
   [junit4]   2> 		at org.apache.solr.core.CloserThread.run(CoreContainer.java:1035)
   [junit4]   2> 	
   [junit4]   2> 	"searcherExecutor-1487-thread-1" ID=4131 WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@18db27c
   [junit4]   2> 		at sun.misc.Unsafe.park(Native Method)
   [junit4]   2> 		- waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@18db27c
   [junit4]   2> 		at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
   [junit4]   2> 		at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
   [junit4]   2> 		at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]   2> 		at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
   [junit4]   2> 		at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]   2> 		at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]   2> 		at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 	
   [junit4]   2> 	"TEST-CursorPagingTest.testBadInputs-seed#[1A319452CA93DFC9]" ID=4129 WAITING on java.util.concurrent.locks.ReentrantReadWriteLock$FairSync@3cae9a12
   [junit4]   2> 		at sun.misc.Unsafe.park(Native Method)
   [junit4]   2> 		- waiting on java.util.concurrent.locks.ReentrantReadWriteLock$FairSync@3cae9a12
   [junit4]   2> 		at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
   [junit4]   2> 		at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
   [junit4]   2> 		at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870)
   [junit4]   2> 		at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199)
   [junit4]   2> 		at java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.lock(ReentrantReadWriteLock.java:943)
   [junit4]   2> 		at org.apache.solr.update.VersionInfo.blockUpdates(VersionInfo.java:119)
   [junit4]   2> 		at org.apache.solr.update.UpdateLog.onFirstSearcher(UpdateLog.java:1604)
   [junit4]   2> 		at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1810)
   [junit4]   2> 		at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1505)
   [junit4]   2> 		at org.apache.solr.update.DirectUpdateHandler2.commit(DirectUpdateHandler2.java:617)
   [junit4]   2> 		- locked java.lang.Object@109072f
   [junit4]   2> 		at org.apache.solr.update.processor.RunUpdateProcessor.processCommit(RunUpdateProcessorFactory.java:95)
   [junit4]   2> 		at org.apache.solr.update.processor.UpdateRequestProcessor.processCommit(UpdateRequestProcessor.java:64)
   [junit4]   2> 		at org.apache.solr.update.processor.DistributedUpdateProcessor.doLocalCommit(DistributedUpdateProcessor.java:1635)
   [junit4]   2> 		at org.apache.solr.update.processor.DistributedUpdateProcessor.processCommit(DistributedUpdateProcessor.java:1612)
   [junit4]   2> 		at org.apache.solr.update.processor.LogUpdateProcessor.processCommit(LogUpdateProcessorFactory.java:161)
   [junit4]   2> 		at org.apache.solr.handler.loader.XMLLoader.processUpdate(XMLLoader.java:270)
   [junit4]   2> 		at org.apache.solr.handler.loader.XMLLoader.load(XMLLoader.java:177)
   [junit4]   2> 		at org.apache.solr.handler.UpdateRequestHandler$1.load(UpdateRequestHandler.java:98)
   [junit4]   2> 		at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:74)
   [junit4]   2> 		at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:143)
   [junit4]   2> 		at org.apache.solr.core.SolrCore.execute(SolrCore.java:2051)
   [junit4]   2> 		at org.apache.solr.servlet.DirectSolrConnection.request(DirectSolrConnection.java:131)
   [junit4]   2> 		at org.apache.solr.util.TestHarness.update(TestHarness.java:267)
   [junit4]   2> 		at org.apache.solr.util.BaseTestHarness.checkUpdateStatus(BaseTestHarness.java:282)
   [junit4]   2> 		at org.apache.solr.util.BaseTestHarness.validateUpdate(BaseTestHarness.java:252)
   [junit4]   2> 		at org.apache.solr.SolrTestCaseJ4.checkUpdateU(SolrTestCaseJ4.java:748)
   [junit4]   2> 		at org.apache.solr.SolrTestCaseJ4.assertU(SolrTestCaseJ4.java:727)
   [junit4]   2> 		at org.apache.solr.SolrTestCaseJ4.assertU(SolrTestCaseJ4.java:721)
   [junit4]   2> 		at org.apache.solr.CursorPagingTest.testBadInputs(CursorPagingTest.java:86)
   [junit4]   2> 		at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   2> 		at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   [junit4]   2> 		at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2> 		at java.lang.reflect.Method.invoke(Method.java:497)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2> 		at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
   [junit4]   2> 		at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2> 		at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
   [junit4]   2> 		at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2> 		at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2> 		at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 		at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 		at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   2> 		at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2> 		at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2> 		at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   2> 		at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 		Locked synchronizers:
   [junit4]   2> 		- java.util.concurrent.locks.ReentrantLock$FairSync@791dd89f
   [junit4]   2> 		- java.util.concurrent.locks.ReentrantLock$NonfairSync@505ccc37
   [junit4]   2> 	
   [junit4]   2> 	"SUITE-CursorPagingTest-seed#[1A319452CA93DFC9]" ID=4128 RUNNABLE
   [junit4]   2> 		at sun.management.ThreadImpl.dumpThreads0(Native Method)
   [junit4]   2> 		at sun.management.ThreadImpl.dumpAllThreads(ThreadImpl.java:446)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.ThreadLeakControl.formatThreadStacksFull(ThreadLeakControl.java:671)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.ThreadLeakControl.access$900(ThreadLeakControl.java:62)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.ThreadLeakControl$2.evaluate(ThreadLeakControl.java:412)
   [junit4]   2> 		- locked java.lang.Object@7557d2d7
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:651)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.RandomizedRunner.access$200(RandomizedRunner.java:138)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.RandomizedRunner$1.run(RandomizedRunner.java:568)
   [junit4]   2> 	
   [junit4]   2> 	"Thread-501" ID=1617 TIMED_WAITING
   [junit4]   2> 		at java.lang.Thread.sleep(Native Method)
   [junit4]   2> 		at org.apache.zookeeper.Login$1.run(Login.java:183)
   [junit4]   2> 		at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 	
   [junit4]   2> 	"JUnit4-serializer-daemon" ID=10 TIMED_WAITING
   [junit4]   2> 		at java.lang.Thread.sleep(Native Method)
   [junit4]   2> 		at com.carrotsearch.ant.tasks.junit4.events.Serializer$1.run(Serializer.java:47)
   [junit4]   2> 	
   [junit4]   2> 	"Attach Listener" ID=5 RUNNABLE
   [junit4]   2> 	
   [junit4]   2> 	"Signal Dispatcher" ID=4 RUNNABLE
   [junit4]   2> 	
   [junit4]   2> 	"Finalizer" ID=3 WAITING on java.lang.ref.ReferenceQueue$Lock@621935c2
   [junit4]   2> 		at java.lang.Object.wait(Native Method)
   [junit4]   2> 		- waiting on java.lang.ref.ReferenceQueue$Lock@621935c2
   [junit4]   2> 		at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:143)
   [junit4]   2> 		at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:164)
   [junit4]   2> 		at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:209)
   [junit4]   2> 	
   [junit4]   2> 	"Reference Handler" ID=2 WAITING on java.lang.ref.Reference$Lock@541270c2
   [junit4]   2> 		at java.lang.Object.wait(Native Method)
   [junit4]   2> 		- waiting on java.lang.ref.Reference$Lock@541270c2
   [junit4]   2> 		at java.lang.Object.wait(Object.java:502)
   [junit4]   2> 		at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:157)
   [junit4]   2> 	
   [junit4]   2> 	"main" ID=1 WAITING on com.carrotsearch.randomizedtesting.RandomizedRunner$1@714e3943
   [junit4]   2> 		at java.lang.Object.wait(Native Method)
   [junit4]   2> 		- waiting on com.carrotsearch.randomizedtesting.RandomizedRunner$1@714e3943
   [junit4]   2> 		at java.lang.Thread.join(Thread.java:1245)
   [junit4]   2> 		at java.lang.Thread.join(Thread.java:1319)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:578)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.RandomizedRunner.run(RandomizedRunner.java:444)
   [junit4]   2> 		at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.execute(SlaveMain.java:199)
   [junit4]   2> 		at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.main(SlaveMain.java:310)
   [junit4]   2> 		at com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe.main(SlaveMainSafe.java:12)
   [junit4]   2> 	
   [junit4]   2> 	^^==============================================
   [junit4]   2> 	
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=CursorPagingTest -Dtests.method=testBadInputs -Dtests.seed=1A319452CA93DFC9 -Dtests.slow=true -Dtests.locale=es_CO -Dtests.timezone=Europe/San_Marino -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
   [junit4] ERROR   7213s J1 | CursorPagingTest.testBadInputs <<<
   [junit4]    > Throwable #1: java.lang.Exception: Test abandoned because suite timeout was reached.
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([1A319452CA93DFC9]:0)
   [junit4]   2> 7214245 T4128 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 3 leaked thread(s).
   [junit4]   2> 7234402 T4128 ccr.ThreadLeakControl.checkThreadLeaks SEVERE 3 threads leaked from SUITE scope at org.apache.solr.CursorPagingTest: 
   [junit4]   2> 	   1) Thread[id=4129, name=TEST-CursorPagingTest.testBadInputs-seed#[1A319452CA93DFC9], state=WAITING, group=TGRP-CursorPagingTest]
   [junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
   [junit4]   2> 	        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
   [junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
   [junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870)
   [junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199)
   [junit4]   2> 	        at java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.lock(ReentrantReadWriteLock.java:943)
   [junit4]   2> 	        at org.apache.solr.update.VersionInfo.blockUpdates(VersionInfo.java:119)
   [junit4]   2> 	        at org.apache.solr.update.UpdateLog.onFirstSearcher(UpdateLog.java:1604)
   [junit4]   2> 	        at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1810)
   [junit4]   2> 	        at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1505)
   [junit4]   2> 	        at org.apache.solr.update.DirectUpdateHandler2.commit(DirectUpdateHandler2.java:617)
   [junit4]   2> 	        at org.apache.solr.update.processor.RunUpdateProcessor.processCommit(RunUpdateProcessorFactory.java:95)
   [junit4]   2> 	        at org.apache.solr.update.processor.UpdateRequestProcessor.processCommit(UpdateRequestProcessor.java:64)
   [junit4]   2> 	        at org.apache.solr.update.processor.DistributedUpdateProcessor.doLocalCommit(DistributedUpdateProcessor.java:1635)
   [junit4]   2> 	        at org.apache.solr.update.processor.DistributedUpdateProcessor.processCommit(DistributedUpdateProcessor.java:1612)
   [junit4]   2> 	        at org.apache.solr.update.processor.LogUpdateProcessor.processCommit(LogUpdateProcessorFactory.java:161)
   [junit4]   2> 	        at org.apache.solr.handler.loader.XMLLoader.processUpdate(XMLLoader.java:270)
   [junit4]   2> 	        at org.apache.solr.handler.loader.XMLLoader.load(XMLLoader.java:177)
   [junit4]   2> 	        at org.apache.solr.handler.UpdateRequestHandler$1.load(UpdateRequestHandler.java:98)
   [junit4]   2> 	        at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:74)
   [junit4]   2> 	        at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:143)
   [junit4]   2> 	        at org.apache.solr.core.SolrCore.execute(SolrCore.java:2051)
   [junit4]   2> 	        at org.apache.solr.servlet.DirectSolrConnection.request(DirectSolrConnection.java:131)
   [junit4]   2> 	        at org.apache.solr.util.TestHarness.update(TestHarness.java:267)
   [junit4]   2> 	        at org.apache.solr.util.BaseTestHarness.checkUpdateStatus(BaseTestHarness.java:282)
   [junit4]   2> 	        at org.apache.solr.util.BaseTestHarness.validateUpdate(BaseTestHarness.java:252)
   [junit4]   2> 	        at org.apache.solr.SolrTestCaseJ4.checkUpdateU(SolrTestCaseJ4.java:748)
   [junit4]   2> 	        at org.apache.solr.SolrTestCaseJ4.assertU(SolrTestCaseJ4.java:727)
   [junit4]   2> 	        at org.apache.solr.SolrTestCaseJ4.assertU(SolrTestCaseJ4.java:721)
   [junit4]   2> 	        at org.apache.solr.CursorPagingTest.testBadInputs(CursorPagingTest.java:86)
   [junit4]   2> 	        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   2> 	        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   [junit4]   2> 	        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2> 	        at java.lang.reflect.Method.invoke(Method.java:497)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2> 	        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
   [junit4]   2> 	        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2> 	        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
   [junit4]   2> 	        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2> 	        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2> 	        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   2> 	        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2> 	        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2> 	        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   2> 	        at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 	   2) Thread[id=4131, name=searcherExecutor-1487-thread-1, state=WAITING, group=TGRP-CursorPagingTest]
   [junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
   [junit4]   2> 	        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
   [junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
   [junit4]   2> 	        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
   [junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]   2> 	        at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 	   3) Thread[id=4132, name=Thread-838, state=WAITING, group=TGRP-CursorPagingTest]
   [junit4]   2> 	        at java.lang.Object.wait(Native Method)
   [junit4]   2> 	        at java.lang.Object.wait(Object.java:502)
   [junit4]   2> 	        at org.apache.solr.core.CloserThread.run(CoreContainer.java:1035)
   [junit4]   2> 7234403 T4128 ccr.ThreadLeakControl.tryToInterruptAll Starting to interrupt leaked threads:
   [junit4]   2> 	   1) Thread[id=4129, name=TEST-CursorPagingTest.testBadInputs-seed#[1A319452CA93DFC9], state=WAITING, group=TGRP-CursorPagingTest]
   [junit4]   2> 	   2) Thread[id=4131, name=searcherExecutor-1487-thread-1, state=WAITING, group=TGRP-CursorPagingTest]
   [junit4]   2> 	   3) Thread[id=4132, name=Thread-838, state=WAITING, group=TGRP-CursorPagingTest]
   [junit4]   2> 7237430 T4128 ccr.ThreadLeakControl.tryToInterruptAll SEVERE There are still zombie threads that couldn't be terminated:
   [junit4]   2> 	   1) Thread[id=4129, name=TEST-CursorPagingTest.testBadInputs-seed#[1A319452CA93DFC9], state=WAITING, group=TGRP-CursorPagingTest]
   [junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
   [junit4]   2> 	        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
   [junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
   [junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870)
   [junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199)
   [junit4]   2> 	        at java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.lock(ReentrantReadWriteLock.java:943)
   [junit4]   2> 	        at org.apache.solr.update.VersionInfo.blockUpdates(VersionInfo.java:119)
   [junit4]   2> 	        at org.apache.solr.update.UpdateLog.onFirstSearcher(UpdateLog.java:1604)
   [junit4]   2> 	        at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1810)
   [junit4]   2> 	        at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1505)
   [junit4]   2> 	        at org.apache.solr.update.DirectUpdateHandler2.commit(DirectUpdateHandler2.java:617)
   [junit4]   2> 	        at org.apache.solr.update.processor.RunUpdateProcessor.processCommit(RunUpdateProcessorFactory.java:95)
   [junit4]   2> 	        at org.apache.solr.update.processor.UpdateRequestProcessor.processCommit(UpdateRequestProcessor.java:64)
   [junit4]   2> 	        at org.apache.solr.update.processor.DistributedUpdateProcessor.doLocalCommit(DistributedUpdateProcessor.java:1635)
   [junit4]   2> 	        at org.apache.solr.update.processor.DistributedUpdateProcessor.processCommit(DistributedUpdateProcessor.java:1612)
   [junit4]   2> 	        at org.apache.solr.update.processor.LogUpdateProcessor.processCommit(LogUpdateProcessorFactory.java:161)
   [junit4]   2> 	        at org.apache.solr.handler.loader.XMLLoader.processUpdate(XMLLoader.java:270)
   [junit4]   2> 	        at org.apache.solr.handler.loader.XMLLoader.load(XMLLoader.java:177)
   [junit4]   2> 	        at org.apache.solr.handler.UpdateRequestHandler$1.load(UpdateRequestHandler.java:98)
   [junit4]   2> 	        at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:74)
   [junit4]   2> 	        at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:143)
   [junit4]   2> 	        at org.apache.solr.core.SolrCore.execute(SolrCore.java:2051)
   [junit4]   2> 	        at org.apache.solr.servlet.DirectSolrConnection.request(DirectSolrConnection.java:131)
   [junit4]   2> 	        at org.apache.solr.util.TestHarness.update(TestHarness.java:267)
   [junit4]   2> 	        at org.apache.solr.util.BaseTestHarness.checkUpdateStatus(BaseTestHarness.java:282)
   [junit4]   2> 	        at org.apache.solr.util.BaseTestHarness.validateUpdate(BaseTestHarness.java:252)
   [junit4]   2> 	        at org.apache.solr.SolrTestCaseJ4.checkUpdateU(SolrTestCaseJ4.java:748)
   [junit4]   2> 	        at org.apache.solr.SolrTestCaseJ4.assertU(SolrTestCaseJ4.java:727)
   [junit4]   2> 	        at org.apache.solr.SolrTestCaseJ4.assertU(SolrTestCaseJ4.java:721)
   [junit4]   2> 	        at org.apache.solr.CursorPagingTest.testBadInputs(CursorPagingTest.java:86)
   [junit4]   2> 	        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   2> 	        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   [junit4]   2> 	        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2> 	        at java.lang.reflect.Method.invoke(Method.java:497)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2> 	        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
   [junit4]   2> 	        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2> 	        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
   [junit4]   2> 	        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2> 	        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2> 	        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   2> 	        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2> 	        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2> 	        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   2> 	        at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 	   2) Thread[id=4131, name=searcherExecutor-1487-thread-1, state=WAITING, group=TGRP-CursorPagingTest]
   [junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
   [junit4]   2> 	        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
   [junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
   [junit4]   2> 	        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
   [junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]   2> 	        at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 	   3) Thread[id=4132, name=Thread-838, state=WAITING, group=TGRP-CursorPagingTest]
   [junit4]   2> 	        at java.lang.Object.wait(Native Method)
   [junit4]   2> 	        at java.lang.Object.wait(Object.java:502)
   [junit4]   2> 	        at org.apache.solr.core.CloserThread.run(CoreContainer.java:1035)
   [junit4]   2> NOTE: test params are: codec=FastCompressingStoredFields(storedFieldsFormat=CompressingStoredFieldsFormat(compressionMode=FAST, chunkSize=22937, maxDocsPerChunk=9, blockSize=6), termVectorsFormat=CompressingTermVectorsFormat(compressionMode=FAST, chunkSize=22937, blockSize=6)), sim=DefaultSimilarity, locale=es_CO, timezone=Europe/San_Marino
   [junit4]   2> NOTE: Windows 7 6.1 amd64/Oracle Corporation 1.8.0_45 (64-bit)/cpus=3,threads=4,free=38080736,total=194854912
   [junit4]   2> NOTE: All tests run in this JVM: [SpatialFilterTest, BJQParserTest, TestDocSet, OverseerStatusTest, MinimalSchemaTest, OpenCloseCoreStressTest, StatelessScriptUpdateProcessorFactoryTest, CopyFieldTest, CoreAdminCreateDiscoverTest, LeaderElectionIntegrationTest, TestFieldTypeResource, DisMaxRequestHandlerTest, TestSchemaVersionResource, TestCodecSupport, PolyFieldTest, DirectSolrSpellCheckerTest, ShardRoutingCustomTest, IgnoreCommitOptimizeUpdateProcessorFactoryTest, MigrateRouteKeyTest, FastVectorHighlighterTest, FacetPivotSmallTest, OpenExchangeRatesOrgProviderTest, ChaosMonkeySafeLeaderTest, SolrRequestParserTest, TestQuerySenderNoQuery, SpellPossibilityIteratorTest, FullSolrCloudDistribCmdsTest, SaslZkACLProviderTest, TestDistributedStatsComponentCardinality, ConvertedLegacyTest, QueryParsingTest, FileUtilsTest, DistributedFacetPivotLongTailTest, TestRecovery, TestShortCircuitedRequests, TestCloudManagedSchemaConcurrent, RuleEngineTest, VMParamsZkACLAndCredentialsProvidersTest, SpatialRPTFieldTypeTest, UpdateParamsTest, CursorPagingTest]
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=CursorPagingTest -Dtests.seed=1A319452CA93DFC9 -Dtests.slow=true -Dtests.locale=es_CO -Dtests.timezone=Europe/San_Marino -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
   [junit4] ERROR   0.00s J1 | CursorPagingTest (suite) <<<
   [junit4]    > Throwable #1: java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([1A319452CA93DFC9]:0)
   [junit4] Completed [414/496] on J1 in 7237.46s, 1 test, 2 errors <<< FAILURES!

[...truncated 264 lines...]
BUILD FAILED
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\build.xml:526: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\build.xml:474: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\build.xml:61: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\extra-targets.xml:39: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build.xml:229: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\common-build.xml:512: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\common-build.xml:1415: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\common-build.xml:973: There were test failures: 496 suites, 1965 tests, 1 suite-level error, 1 error, 327 ignored (33 assumptions)

Total time: 145 minutes 49 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any