You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Policeman Jenkins Server <je...@thetaphi.de> on 2014/02/25 21:05:07 UTC

[JENKINS] Lucene-Solr-trunk-Windows (64bit/jdk1.8.0-fcs-b129) - Build # 3817 - Still Failing!

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Windows/3817/
Java: 64bit/jdk1.8.0-fcs-b129 -XX:-UseCompressedOops -XX:+UseConcMarkSweepGC

2 tests failed.
REGRESSION:  org.apache.solr.core.TestLazyCores.testLazyLoad

Error Message:
Test abandoned because suite timeout was reached.

Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
	at __randomizedtesting.SeedInfo.seed([1E468F4F611387A5]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.core.TestLazyCores

Error Message:
Suite timeout exceeded (>= 7200000 msec).

Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
	at __randomizedtesting.SeedInfo.seed([1E468F4F611387A5]:0)




Build Log:
[...truncated 10682 lines...]
   [junit4] Suite: org.apache.solr.core.TestLazyCores
   [junit4]   2> 2647583 T9805 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (false) and clientAuth (true)
   [junit4]   2> 2647583 T9805 oas.SolrTestCaseJ4.initCore ####initCore
   [junit4]   2> Creating dataDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\solrtest-TestLazyCores-1393351351495
   [junit4]   2> 2647585 T9805 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\'
   [junit4]   2> 2647587 T9805 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/C:/Users/JenkinsSlave/workspace/Lucene-Solr-trunk-Windows/solr/core/src/test-files/solr/collection1/lib/.svn/' to classloader
   [junit4]   2> 2647588 T9805 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/C:/Users/JenkinsSlave/workspace/Lucene-Solr-trunk-Windows/solr/core/src/test-files/solr/collection1/lib/classes/' to classloader
   [junit4]   2> 2647588 T9805 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/C:/Users/JenkinsSlave/workspace/Lucene-Solr-trunk-Windows/solr/core/src/test-files/solr/collection1/lib/README' to classloader
   [junit4]   2> 2647625 T9805 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
   [junit4]   2> 2647633 T9805 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig-minimal.xml
   [junit4]   2> 2647633 T9805 oass.IndexSchema.readSchema Reading Solr Schema from schema-tiny.xml
   [junit4]   2> 2647636 T9805 oass.IndexSchema.readSchema [null] Schema name=tiny
   [junit4]   2> 2647641 T9805 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 2647641 T9805 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 2647641 T9805 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr
   [junit4]   2> 2647641 T9805 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\'
   [junit4]   2> 2647660 T9805 oasc.CoreContainer.<init> New CoreContainer 341627132
   [junit4]   2> 2647660 T9805 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\]
   [junit4]   2> 2647662 T9805 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
   [junit4]   2> 2647662 T9805 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: null
   [junit4]   2> 2647662 T9805 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
   [junit4]   2> 2647662 T9805 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 2647662 T9805 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 2647662 T9805 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 2647662 T9805 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 2647662 T9805 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 2647662 T9805 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 2647666 T9805 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 2647667 T9805 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 2647667 T9805 oasc.CoreContainer.load Host Name: 
   [junit4]   2> 2647670 T9806 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1
   [junit4]   2> 2647670 T9806 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\'
   [junit4]   2> 2647670 T9806 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/C:/Users/JenkinsSlave/workspace/Lucene-Solr-trunk-Windows/solr/core/src/test-files/solr/collection1/lib/.svn/' to classloader
   [junit4]   2> 2647670 T9806 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/C:/Users/JenkinsSlave/workspace/Lucene-Solr-trunk-Windows/solr/core/src/test-files/solr/collection1/lib/classes/' to classloader
   [junit4]   2> 2647670 T9806 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/C:/Users/JenkinsSlave/workspace/Lucene-Solr-trunk-Windows/solr/core/src/test-files/solr/collection1/lib/README' to classloader
   [junit4]   2> 2647695 T9806 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
   [junit4]   2> 2647702 T9806 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig-minimal.xml
   [junit4]   2> 2647702 T9806 oass.IndexSchema.readSchema Reading Solr Schema from schema-tiny.xml
   [junit4]   2> 2647706 T9806 oass.IndexSchema.readSchema [collection1] Schema name=tiny
   [junit4]   2> 2647708 T9806 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 2647708 T9806 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 2647711 T9806 oasc.SolrCore.<init> [collection1] Opening new SolrCore at C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\, dataDir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\solrtest-TestLazyCores-1393351351495\
   [junit4]   2> 2647711 T9806 oasc.SolrCore.<init> JMX monitoring not detected for core: collection1
   [junit4]   2> 2647714 T9806 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\solrtest-TestLazyCores-1393351351495\
   [junit4]   2> 2647714 T9806 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\solrtest-TestLazyCores-1393351351495\index/
   [junit4]   2> 2647715 T9806 oasc.SolrCore.initIndex WARN [collection1] Solr index directory 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\solrtest-TestLazyCores-1393351351495\index' doesn't exist. Creating new index...
   [junit4]   2> 2647716 T9806 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\solrtest-TestLazyCores-1393351351495\index
   [junit4]   2> 2647716 T9806 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=39, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.3628029192786042]
   [junit4]   2> 2647717 T9806 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@2cef803f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@324d079f),segFN=segments_1,generation=1}
   [junit4]   2> 2647717 T9806 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 2647718 T9806 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 2647718 T9806 oasc.RequestHandlers.initHandlersFromConfig created /select: solr.SearchHandler
   [junit4]   2> 2647719 T9806 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 2647719 T9806 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 2647724 T9806 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 2647725 T9806 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 2647725 T9806 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 2647725 T9806 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=37, maxMergeAtOnceExplicit=43, maxMergedSegmentMB=85.0830078125, floorSegmentMB=1.7626953125, forceMergeDeletesPctAllowed=29.124094156630637, segmentsPerTier=23.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.600800832000014
   [junit4]   2> 2647725 T9806 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@2cef803f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@324d079f),segFN=segments_1,generation=1}
   [junit4]   2> 2647725 T9806 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 2647725 T9806 oass.SolrIndexSearcher.<init> Opening Searcher@5d3a45c7[collection1] main
   [junit4]   2> 2647727 T9806 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 2647727 T9807 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5d3a45c7[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
   [junit4]   2> 2647729 T9805 oas.SolrTestCaseJ4.initCore ####initCore end
   [junit4]   2> 2647735 T9805 oas.SolrTestCaseJ4.setUp ###Starting testRace
   [junit4]   2> 2647801 T9805 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\'
   [junit4]   2> 2647827 T9805 oasc.ConfigSolr.fromFile Loading container configuration from C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\solr.xml
   [junit4]   2> 2647836 T9805 oasc.CoreContainer.<init> New CoreContainer 1629219968
   [junit4]   2> 2647836 T9805 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\]
   [junit4]   2> 2647837 T9805 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
   [junit4]   2> 2647837 T9805 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: null
   [junit4]   2> 2647837 T9805 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
   [junit4]   2> 2647837 T9805 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 2647838 T9805 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 2647838 T9805 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 2647838 T9805 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 2647838 T9805 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 2647838 T9805 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 2647840 T9805 oasc.SolrCores.allocateLazyCores Allocating transient cache for 4 transient cores
   [junit4]   2> 2647840 T9805 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 2647840 T9805 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 2647840 T9805 oasc.CoreContainer.load Host Name: null
   [junit4]   2> 2647852 T9809 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\collection1
   [junit4]   2> 2647852 T9809 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\collection1\'
   [junit4]   2> 2647876 T9809 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
   [junit4]   2> 2647882 T9809 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 2647882 T9809 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 2647887 T9809 oass.IndexSchema.readSchema [collection1] Schema name=tiny
   [junit4]   2> 2647891 T9809 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 2647891 T9809 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 2647893 T9809 oasc.SolrCore.<init> [collection1] Opening new SolrCore at C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\collection1\, dataDir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\collection1\data\
   [junit4]   2> 2647893 T9809 oasc.SolrCore.<init> JMX monitoring not detected for core: collection1
   [junit4]   2> 2647894 T9809 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\collection1\data\
   [junit4]   2> 2647894 T9809 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\collection1\data\index/
   [junit4]   2> 2647894 T9809 oasc.SolrCore.initIndex WARN [collection1] Solr index directory 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\collection1\data\index' doesn't exist. Creating new index...
   [junit4]   2> 2647895 T9809 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\collection1\data\index
   [junit4]   2> 2647895 T9809 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=39, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.3628029192786042]
   [junit4]   2> 2647912 T9809 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@5119a4c lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@41a987fe),segFN=segments_1,generation=1}
   [junit4]   2> 2647912 T9809 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 2647914 T9809 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 2647914 T9809 oasc.RequestHandlers.initHandlersFromConfig created /select: solr.SearchHandler
   [junit4]   2> 2647914 T9809 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 2647915 T9809 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 2647920 T9809 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 2647921 T9809 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 2647921 T9809 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 2647921 T9809 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=37, maxMergeAtOnceExplicit=43, maxMergedSegmentMB=85.0830078125, floorSegmentMB=1.7626953125, forceMergeDeletesPctAllowed=29.124094156630637, segmentsPerTier=23.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.600800832000014
   [junit4]   2> 2647922 T9809 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@5119a4c lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@41a987fe),segFN=segments_1,generation=1}
   [junit4]   2> 2647922 T9809 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 2647922 T9809 oass.SolrIndexSearcher.<init> Opening Searcher@554ec5bd[collection1] main
   [junit4]   2> 2647923 T9809 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 2647923 T9810 oasc.CoreContainer.create Creating SolrCore 'collectionLazy2' using instanceDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\collection2
   [junit4]   2> 2647923 T9810 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\collection2\'
   [junit4]   2> 2647942 T9810 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
   [junit4]   2> 2647947 T9810 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 2647949 T9810 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 2647953 T9810 oass.IndexSchema.readSchema [collectionLazy2] Schema name=tiny
   [junit4]   2> 2647955 T9810 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 2647955 T9810 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 2647957 T9810 oasc.SolrCore.<init> [collectionLazy2] Opening new SolrCore at C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\collection2\, dataDir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\collection2\data\
   [junit4]   2> 2647957 T9810 oasc.SolrCore.<init> JMX monitoring not detected for core: collectionLazy2
   [junit4]   2> 2647957 T9811 oasc.CoreContainer.create Creating SolrCore 'collectionLazy5' using instanceDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\collection5
   [junit4]   2> 2647957 T9811 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\collection5\'
   [junit4]   2> 2647977 T9811 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
   [junit4]   2> 2647983 T9811 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 2647983 T9811 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 2647986 T9811 oass.IndexSchema.readSchema [collectionLazy5] Schema name=tiny
   [junit4]   2> 2647989 T9811 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 2647989 T9811 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 2647989 T9811 oasc.SolrCore.<init> [collectionLazy5] Opening new SolrCore at C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\collection5\, dataDir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\collection5\data\
   [junit4]   2> 2647989 T9811 oasc.SolrCore.<init> JMX monitoring not detected for core: collectionLazy5
   [junit4]   2> 2647990 T9810 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\collection2\data\
   [junit4]   2> 2647990 T9810 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\collection2\data\index/
   [junit4]   2> 2647990 T9810 oasc.SolrCore.initIndex WARN [collectionLazy2] Solr index directory 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\collection2\data\index' doesn't exist. Creating new index...
   [junit4]   2> 2647991 T9811 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\collection5\data\
   [junit4]   2> 2647991 T9811 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\collection5\data\index/
   [junit4]   2> 2647991 T9811 oasc.SolrCore.initIndex WARN [collectionLazy5] Solr index directory 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\collection5\data\index' doesn't exist. Creating new index...
   [junit4]   2> 2647992 T9810 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\collection2\data\index
   [junit4]   2> 2647992 T9810 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=39, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.3628029192786042]
   [junit4]   2> 2647993 T9811 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\collection5\data\index
   [junit4]   2> 2647993 T9811 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=39, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.3628029192786042]
   [junit4]   2> 2647994 T9810 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@59a9f1c2 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2a4b311),segFN=segments_1,generation=1}
   [junit4]   2> 2647994 T9810 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 2647994 T9811 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@74f89444 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@747f5054),segFN=segments_1,generation=1}
   [junit4]   2> 2647994 T9811 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 2647995 T9810 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 2647995 T9810 oasc.RequestHandlers.initHandlersFromConfig created /select: solr.SearchHandler
   [junit4]   2> 2647995 T9810 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 2647995 T9810 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 2647997 T9810 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 2647998 T9811 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 2647999 T9811 oasc.RequestHandlers.initHandlersFromConfig created /select: solr.SearchHandler
   [junit4]   2> 2647999 T9811 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 2647999 T9811 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 2648001 T9811 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 2648001 T9810 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 2648001 T9810 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 2648002 T9811 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 2648002 T9811 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 2648002 T9810 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=37, maxMergeAtOnceExplicit=43, maxMergedSegmentMB=85.0830078125, floorSegmentMB=1.7626953125, forceMergeDeletesPctAllowed=29.124094156630637, segmentsPerTier=23.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.600800832000014
   [junit4]   2> 2648002 T9811 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=37, maxMergeAtOnceExplicit=43, maxMergedSegmentMB=85.0830078125, floorSegmentMB=1.7626953125, forceMergeDeletesPctAllowed=29.124094156630637, segmentsPerTier=23.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.600800832000014
   [junit4]   2> 2648003 T9810 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@59a9f1c2 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2a4b311),segFN=segments_1,generation=1}
   [junit4]   2> 2648003 T9810 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 2648003 T9810 oass.SolrIndexSearcher.<init> Opening Searcher@30f4cc78[collectionLazy2] main
   [junit4]   2> 2648004 T9811 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@74f89444 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@747f5054),segFN=segments_1,generation=1}
   [junit4]   2> 2648005 T9811 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 2648005 T9811 oass.SolrIndexSearcher.<init> Opening Searcher@796e5c82[collectionLazy5] main
   [junit4]   2> 2648006 T9813 oasc.SolrCore.registerSearcher [collectionLazy2] Registered new searcher Searcher@30f4cc78[collectionLazy2] main{StandardDirectoryReader(segments_1:1:nrt)}
   [junit4]   2> 2648005 T9810 oasc.SolrCores.putTransientCore Opening transient core collectionLazy2
   [junit4]   2> 2648006 T9810 oasc.CoreContainer.registerCore registering core: collectionLazy2
   [junit4]   2> 2648006 T9811 oasc.CoreContainer.registerCore registering core: collectionLazy5
   [junit4]   2> 2648008 T9812 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@554ec5bd[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
   [junit4]   2> 2648009 T9814 oasc.SolrCore.registerSearcher [collectionLazy5] Registered new searcher Searcher@796e5c82[collectionLazy5] main{StandardDirectoryReader(segments_1:1:nrt)}
   [junit4]   2> 2648011 T9816 oasc.CoreContainer.create Creating SolrCore 'collectionLazy3' using instanceDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\collection3
   [junit4]   2> 2648011 T9816 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\collection3\'
   [junit4]   2> 2648029 T9816 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
   [junit4]   2> 2648047 T9816 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 2648049 T9816 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 2648052 T9816 oass.IndexSchema.readSchema [collectionLazy3] Schema name=tiny
   [junit4]   2> 2648055 T9816 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 2648055 T9816 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 2648057 T9816 oasc.SolrCore.<init> [collectionLazy3] Opening new SolrCore at C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\collection3\, dataDir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\collection3\data\
   [junit4]   2> 2648057 T9816 oasc.SolrCore.<init> JMX monitoring not detected for core: collectionLazy3
   [junit4]   2> 2648073 T9816 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\collection3\data\
   [junit4]   2> 2648073 T9816 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\collection3\data\index/
   [junit4]   2> 2648073 T9816 oasc.SolrCore.initIndex WARN [collectionLazy3] Solr index directory 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\collection3\data\index' doesn't exist. Creating new index...
   [junit4]   2> 2648074 T9816 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\collection3\data\index
   [junit4]   2> 2648075 T9816 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=39, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.3628029192786042]
   [junit4]   2> 2648075 T9816 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@62780ff9 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@42b4d86b),segFN=segments_1,generation=1}
   [junit4]   2> 2648075 T9816 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 2648077 T9816 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 2648078 T9816 oasc.RequestHandlers.initHandlersFromConfig created /select: solr.SearchHandler
   [junit4]   2> 2648078 T9816 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 2648078 T9816 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 2648080 T9816 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 2648082 T9816 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 2648082 T9816 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 2648083 T9816 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=37, maxMergeAtOnceExplicit=43, maxMergedSegmentMB=85.0830078125, floorSegmentMB=1.7626953125, forceMergeDeletesPctAllowed=29.124094156630637, segmentsPerTier=23.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.600800832000014
   [junit4]   2> 2648083 T9816 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@62780ff9 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@42b4d86b),segFN=segments_1,generation=1}
   [junit4]   2> 2648083 T9816 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 2648083 T9816 oass.SolrIndexSearcher.<init> Opening Searcher@63c33ffe[collectionLazy3] main
   [junit4]   2> 2648085 T9816 oasc.SolrCores.putTransientCore Opening transient core collectionLazy3
   [junit4]   2> 2648085 T9816 oasc.CoreContainer.registerCore registering core: collectionLazy3
   [junit4]   2> 2648086 T9831 oasc.SolrCore.registerSearcher [collectionLazy3] Registered new searcher Searcher@63c33ffe[collectionLazy3] main{StandardDirectoryReader(segments_1:1:nrt)}
   [junit4]   2> 2648094 T9805 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=1629219968
   [junit4]   2> 2648095 T9805 oasc.SolrCore.close [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@48a4ec42
   [junit4]   2> 2648095 T9805 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
   [junit4]   2> 2648095 T9805 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
   [junit4]   2> 2648095 T9805 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
   [junit4]   2> 2648095 T9805 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
   [junit4]   2> 2648097 T9805 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
   [junit4]   2> 2648097 T9805 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
   [junit4]   2> 2648097 T9805 oasc.CachingDirectoryFactory.closeCacheValue looking to close C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\collection1\data\index [CachedDir<<refCount=0;path=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\collection1\data\index;done=false>>]
   [junit4]   2> 2648097 T9805 oasc.CachingDirectoryFactory.close Closing directory: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\collection1\data\index
   [junit4]   2> 2648097 T9805 oasc.CachingDirectoryFactory.closeCacheValue looking to close C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\collection1\data\ [CachedDir<<refCount=0;path=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\collection1\data\;done=false>>]
   [junit4]   2> 2648097 T9805 oasc.CachingDirectoryFactory.close Closing directory: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\collection1\data\
   [junit4]   2> 2648097 T9805 oasc.SolrCore.close [collectionLazy5]  CLOSING SolrCore org.apache.solr.core.SolrCore@58b8e950
   [junit4]   2> 2648097 T9805 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
   [junit4]   2> 2648097 T9805 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
   [junit4]   2> 2648097 T9805 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
   [junit4]   2> 2648097 T9805 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
   [junit4]   2> 2648100 T9805 oasc.SolrCore.closeSearcher [collectionLazy5] Closing main searcher on request.
   [junit4]   2> 2648100 T9805 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
   [junit4]   2> 2648100 T9805 oasc.CachingDirectoryFactory.closeCacheValue looking to close C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\collection5\data\index [CachedDir<<refCount=0;path=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\collection5\data\index;done=false>>]
   [junit4]   2> 2648100 T9805 oasc.CachingDirectoryFactory.close Closing directory: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\collection5\data\index
   [junit4]   2> 2648101 T9805 oasc.CachingDirectoryFactory.closeCacheValue looking to close C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\collection5\data\ [CachedDir<<refCount=0;path=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\collection5\data\;done=false>>]
   [junit4]   2> 2648101 T9805 oasc.CachingDirectoryFactory.close Closing directory: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\collection5\data\
   [junit4]   2> 2648101 T9805 oasc.SolrCore.close [collectionLazy2]  CLOSING SolrCore org.apache.solr.core.SolrCore@4263a25
   [junit4]   2> 2648101 T9805 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
   [junit4]   2> 2648101 T9805 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
   [junit4]   2> 2648101 T9805 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
   [junit4]   2> 2648101 T9805 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
   [junit4]   2> 2648103 T9805 oasc.SolrCore.closeSearcher [collectionLazy2] Closing main searcher on request.
   [junit4]   2> 2648103 T9805 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
   [junit4]   2> 2648103 T9805 oasc.CachingDirectoryFactory.closeCacheValue looking to close C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\collection2\data\index [CachedDir<<refCount=0;path=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\collection2\data\index;done=false>>]
   [junit4]   2> 2648103 T9805 oasc.CachingDirectoryFactory.close Closing directory: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\collection2\data\index
   [junit4]   2> 2648104 T9805 oasc.CachingDirectoryFactory.closeCacheValue looking to close C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\collection2\data\ [CachedDir<<refCount=0;path=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\collection2\data\;done=false>>]
   [junit4]   2> 2648104 T9805 oasc.CachingDirectoryFactory.close Closing directory: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\collection2\data\
   [junit4]   2> 2648104 T9805 oasc.SolrCore.close [collectionLazy3]  CLOSING SolrCore org.apache.solr.core.SolrCore@27d4931
   [junit4]   2> 2648104 T9805 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
   [junit4]   2> 2648104 T9805 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
   [junit4]   2> 2648104 T9805 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
   [junit4]   2> 2648104 T9805 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
   [junit4]   2> 2648106 T9805 oasc.SolrCore.closeSearcher [collectionLazy3] Closing main searcher on request.
   [junit4]   2> 2648106 T9805 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
   [junit4]   2> 2648106 T9805 oasc.CachingDirectoryFactory.closeCacheValue looking to close C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\collection3\data\index [CachedDir<<refCount=0;path=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\collection3\data\index;done=false>>]
   [junit4]   2> 2648106 T9805 oasc.CachingDirectoryFactory.close Closing directory: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\collection3\data\index
   [junit4]   2> 2648107 T9805 oasc.CachingDirectoryFactory.closeCacheValue looking to close C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\collection3\data\ [CachedDir<<refCount=0;path=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\collection3\data\;done=false>>]
   [junit4]   2> 2648107 T9805 oasc.CachingDirectoryFactory.close Closing directory: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\collection3\data\
   [junit4]   2> 2648179 T9805 oas.SolrTestCaseJ4.tearDown ###Ending testRace
   [junit4]   2> 2648182 T9805 oas.SolrTestCaseJ4.setUp ###Starting testBadConfigsGenerateErrors
   [junit4]   2> 2648227 T9805 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\'
   [junit4]   2> 2648252 T9805 oasc.ConfigSolr.fromFile Loading container configuration from C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\solr.xml
   [junit4]   2> 2648262 T9805 oasc.CorePropertiesLocator.<init> Config-defined core root directory: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores
   [junit4]   2> 2648263 T9805 oasc.CorePropertiesLocator.<init> Config-defined core root directory: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores
   [junit4]   2> 2648263 T9805 oasc.CoreContainer.<init> New CoreContainer 66674091
   [junit4]   2> 2648263 T9805 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\]
   [junit4]   2> 2648264 T9805 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
   [junit4]   2> 2648264 T9805 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: null
   [junit4]   2> 2648264 T9805 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
   [junit4]   2> 2648264 T9805 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 2648264 T9805 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 2648264 T9805 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 2648265 T9805 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 2648265 T9805 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 2648265 T9805 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 2648269 T9805 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 2648269 T9805 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 2648269 T9805 oasc.CoreContainer.load Host Name: null
   [junit4]   2> 2648270 T9805 oasc.CorePropertiesLocator.discover Looking for core definitions underneath C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores
   [junit4]   2> 2648273 T9805 oasc.CorePropertiesLocator.discoverUnder Found core badConfig1 in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\badConfig1\
   [junit4]   2> 2648273 T9805 oasc.CorePropertiesLocator.discoverUnder Found core badConfig2 in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\badConfig2\
   [junit4]   2> 2648273 T9805 oasc.CorePropertiesLocator.discoverUnder Found core badSchema1 in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\badSchema1\
   [junit4]   2> 2648276 T9805 oasc.CorePropertiesLocator.discoverUnder Found core badSchema2 in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\badSchema2\
   [junit4]   2> 2648276 T9805 oasc.CorePropertiesLocator.discoverUnder Found core core1 in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\core1\
   [junit4]   2> 2648276 T9805 oasc.CorePropertiesLocator.discoverUnder Found core core2 in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\core2\
   [junit4]   2> 2648279 T9805 oasc.CorePropertiesLocator.discover Found 6 core definitions
   [junit4]   2> 2648281 T9833 oasc.CoreContainer.create Creating SolrCore 'badConfig1' using instanceDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\badConfig1
   [junit4]   2> 2648281 T9833 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\badConfig1\'
   [junit4]   2> 2648304 T9833 oasc.SolrException.log Ignoring exception matching \QSAXParseException\E
   [junit4]   2> 2648304 T9833 oasc.CoreContainer.createFromLocal ERROR Failed to load file C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\badConfig1\solrconfig.xml
   [junit4]   2> 2648306 T9833 oasc.CoreContainer.recordAndThrow ERROR Unable to create core: badConfig1 Ignoring exception matching \QSAXParseException\E
   [junit4]   2> 2648306 T9833 oasc.SolrException.log Ignoring exception matching \QSAXParseException\E
   [junit4]   2> 2648307 T9833 oasc.CoreContainer.create Creating SolrCore 'badSchema2' using instanceDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\badSchema2
   [junit4]   2> 2648307 T9833 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\badSchema2\'
   [junit4]   2> 2648324 T9833 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
   [junit4]   2> 2648330 T9833 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 2648330 T9833 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 2648332 T9833 oasc.SolrException.log Ignoring exception matching \QSAXParseException\E
   [junit4]   2> 2648332 T9833 oasc.CoreContainer.recordAndThrow ERROR Unable to create core: badSchema2 Ignoring exception matching \QSAXParseException\E
   [junit4]   2> 2648336 T9833 oasc.SolrException.log Ignoring exception matching \QSAXParseException\E
   [junit4]   2> 2648336 T9833 oasc.CoreContainer.create Creating SolrCore 'core1' using instanceDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\core1
   [junit4]   2> 2648336 T9833 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\core1\'
   [junit4]   2> 2648354 T9833 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
   [junit4]   2> 2648366 T9834 oasc.CoreContainer.create Creating SolrCore 'badConfig2' using instanceDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\badConfig2
   [junit4]   2> 2648366 T9834 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\badConfig2\'
   [junit4]   2> 2648381 T9834 oasc.SolrException.log Ignoring exception matching \QSAXParseException\E
   [junit4]   2> 2648381 T9834 oasc.CoreContainer.createFromLocal ERROR Failed to load file C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\badConfig2\solrconfig.xml
   [junit4]   2> 2648381 T9834 oasc.CoreContainer.recordAndThrow ERROR Unable to create core: badConfig2 Ignoring exception matching \QSAXParseException\E
   [junit4]   2> 2648381 T9834 oasc.SolrException.log Ignoring exception matching \QSAXParseException\E
   [junit4]   2> 2648381 T9834 oasc.CoreContainer.create Creating SolrCore 'core2' using instanceDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\core2
   [junit4]   2> 2648381 T9834 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\core2\'
   [junit4]   2> 2648402 T9834 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
   [junit4]   2> 2648407 T9834 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 2648408 T9834 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 2648412 T9834 oass.IndexSchema.readSchema [core2] Schema name=tiny
   [junit4]   2> 2648416 T9834 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 2648416 T9834 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 2648418 T9834 oasc.SolrCore.<init> [core2] Opening new SolrCore at C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\core2\, dataDir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\core2\data\
   [junit4]   2> 2648418 T9834 oasc.SolrCore.<init> JMX monitoring not detected for core: core2
   [junit4]   2> 2648418 T9834 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\core2\data\
   [junit4]   2> 2648418 T9834 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\core2\data\index/
   [junit4]   2> 2648418 T9834 oasc.SolrCore.initIndex WARN [core2] Solr index directory 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\core2\data\index' doesn't exist. Creating new index...
   [junit4]   2> 2648419 T9834 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\core2\data\index
   [junit4]   2> 2648420 T9834 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=39, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.3628029192786042]
   [junit4]   2> 2648420 T9834 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@17c4de80 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@620406bb),segFN=segments_1,generation=1}
   [junit4]   2> 2648420 T9834 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 2648420 T9834 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 2648420 T9834 oasc.RequestHandlers.initHandlersFromConfig created /select: solr.SearchHandler
   [junit4]   2> 2648421 T9834 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 2648421 T9834 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 2648426 T9834 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 2648427 T9834 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 2648427 T9834 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 2648427 T9834 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=37, maxMergeAtOnceExplicit=43, maxMergedSegmentMB=85.0830078125, floorSegmentMB=1.7626953125, forceMergeDeletesPctAllowed=29.124094156630637, segmentsPerTier=23.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.600800832000014
   [junit4]   2> 2648428 T9834 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@17c4de80 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@620406bb),segFN=segments_1,generation=1}
   [junit4]   2> 2648428 T9834 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 2648428 T9834 oass.SolrIndexSearcher.<init> Opening Searcher@60aead98[core2] main
   [junit4]   2> 2648432 T9833 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 2648438 T9833 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 2648442 T9833 oass.IndexSchema.readSchema [core1] Schema name=tiny
   [junit4]   2> 2648444 T9833 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 2648444 T9833 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 2648445 T9833 oasc.SolrCore.<init> [core1] Opening new SolrCore at C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\core1\, dataDir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\core1\data\
   [junit4]   2> 2648445 T9833 oasc.SolrCore.<init> JMX monitoring not detected for core: core1
   [junit4]   2> 2648438 T9836 oasc.SolrCore.registerSearcher [core2] Registered new searcher Searcher@60aead98[core2] main{StandardDirectoryReader(segments_1:1:nrt)}
   [junit4]   2> 2648438 T9834 oasc.CoreContainer.registerCore registering core: core2
   [junit4]   2> 2648447 T9833 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\core1\data\
   [junit4]   2> 2648447 T9833 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\core1\data\index/
   [junit4]   2> 2648447 T9833 oasc.SolrCore.initIndex WARN [core1] Solr index directory 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\core1\data\index' doesn't exist. Creating new index...
   [junit4]   2> 2648448 T9833 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\core1\data\index
   [junit4]   2> 2648448 T9833 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=39, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.3628029192786042]
   [junit4]   2> 2648448 T9833 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@3e7bdc38 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@294cd809),segFN=segments_1,generation=1}
   [junit4]   2> 2648448 T9833 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 2648449 T9833 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 2648449 T9833 oasc.RequestHandlers.initHandlersFromConfig created /select: solr.SearchHandler
   [junit4]   2> 2648450 T9833 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 2648450 T9833 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 2648451 T9833 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 2648433 T9835 oasc.CoreContainer.create Creating SolrCore 'badSchema1' using instanceDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\badSchema1
   [junit4]   2> 2648451 T9835 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\badSchema1\'
   [junit4]   2> 2648469 T9835 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
   [junit4]   2> 2648475 T9835 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 2648475 T9835 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 2648478 T9835 oasc.SolrException.log Ignoring exception matching \QSAXParseException\E
   [junit4]   2> 2648478 T9835 oasc.CoreContainer.recordAndThrow ERROR Unable to create core: badSchema1 Ignoring exception matching \QSAXParseException\E
   [junit4]   2> 2648480 T9835 oasc.SolrException.log Ignoring exception matching \QSAXParseException\E
   [junit4]   2> 2648480 T9833 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 2648480 T9833 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 2648481 T9833 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=37, maxMergeAtOnceExplicit=43, maxMergedSegmentMB=85.0830078125, floorSegmentMB=1.7626953125, forceMergeDeletesPctAllowed=29.124094156630637, segmentsPerTier=23.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.600800832000014
   [junit4]   2> 2648481 T9833 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@3e7bdc38 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@294cd809),segFN=segments_1,generation=1}
   [junit4]   2> 2648481 T9833 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 2648481 T9833 oass.SolrIndexSearcher.<init> Opening Searcher@220114a6[core1] main
   [junit4]   2> 2648483 T9833 oasc.CoreContainer.registerCore registering core: core1
   [junit4]   2> 2648483 T9837 oasc.SolrCore.registerSearcher [core1] Registered new searcher Searcher@220114a6[core1] main{StandardDirectoryReader(segments_1:1:nrt)}
   [junit4]   2> 2648485 T9805 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 2648485 T9805 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@3e7bdc38 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@294cd809),segFN=segments_1,generation=1}
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@3e7bdc38 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@294cd809),segFN=segments_2,generation=2}
   [junit4]   2> 2648487 T9805 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
   [junit4]   2> 2648487 T9805 oass.SolrIndexSearcher.<init> Opening Searcher@7e5a8a94[core1] main
   [junit4]   2> 2648487 T9805 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 2648488 T9837 oasc.SolrCore.registerSearcher [core1] Registered new searcher Searcher@7e5a8a94[core1] main{StandardDirectoryReader(segments_2:3:nrt _0(5.0):c7)}
   [junit4]   2> ASYNC  NEW_CORE C4366 name=core1 org.apache.solr.core.SolrCore@75de95c1
   [junit4]   2> 2648489 T9805 C4366 oasc.SolrCore.execute [collection1] webapp=null path=null params={q={!prefix+f%3Dv_t}hel&wt=xml} hits=2 status=0 QTime=1 
   [junit4]   2> 2648491 T9805 C4366 oasc.SolrCore.execute [collection1] webapp=null path=null params={q={!raw+f%3Dv_t}hello&wt=xml} hits=2 status=0 QTime=1 
   [junit4]   2> 2648492 T9805 C4366 oasc.SolrCore.execute [collection1] webapp=null path=null params={q={!raw+f%3Dv_t}Hello&wt=xml} hits=0 status=0 QTime=1 
   [junit4]   2> 2648492 T9805 C4366 oasc.SolrCore.execute [collection1] webapp=null path=null params={q={!raw+f%3Dv_f}1.5&wt=xml} hits=0 status=0 QTime=0 
   [junit4]   2> 2648499 T9805 oasc.CoreContainer.create Creating SolrCore 'badConfig1' using instanceDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\badConfig1
   [junit4]   2> 2648499 T9805 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\badConfig1\'
   [junit4]   2> 2648527 T9805 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
   [junit4]   2> 2648534 T9805 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 2648534 T9805 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 2648537 T9805 oass.IndexSchema.readSchema [badConfig1] Schema name=tiny
   [junit4]   2> 2648539 T9805 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 2648539 T9805 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 2648541 T9805 oasc.SolrCore.<init> [badConfig1] Opening new SolrCore at C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\badConfig1\, dataDir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\badConfig1\data\
   [junit4]   2> 2648541 T9805 oasc.SolrCore.<init> JMX monitoring not detected for core: badConfig1
   [junit4]   2> 2648541 T9805 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\badConfig1\data\
   [junit4]   2> 2648541 T9805 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\badConfig1\data\index/
   [junit4]   2> 2648541 T9805 oasc.SolrCore.initIndex WARN [badConfig1] Solr index directory 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\badConfig1\data\index' doesn't exist. Creating new index...
   [junit4]   2> 2648543 T9805 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\badConfig1\data\index
   [junit4]   2> 2648543 T9805 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=20, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.3839131811998765]
   [junit4]   2> 2648543 T9805 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@6156d19f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1aa126c),segFN=segments_1,generation=1}
   [junit4]   2> 2648543 T9805 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 2648545 T9805 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 2648545 T9805 oasc.RequestHandlers.initHandlersFromConfig created /select: solr.SearchHandler
   [junit4]   2> 2648546 T9805 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 2648546 T9805 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 2648547 T9805 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 2648548 T9805 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 2648548 T9805 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 2648548 T9805 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=1083395026, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
   [junit4]   2> 2648549 T9805 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@6156d19f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1aa126c),segFN=segments_1,generation=1}
   [junit4]   2> 2648549 T9805 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 2648549 T9805 oass.SolrIndexSearcher.<init> Opening Searcher@4d83802e[badConfig1] main
   [junit4]   2> 2648551 T9805 oasc.SolrCores.putTransientCore Opening transient core badConfig1
   [junit4]   2> 2648551 T9805 oasc.CoreContainer.registerCore registering core: badConfig1
   [junit4]   2> 2648551 T9805 oasc.CoreContainer.create Creating SolrCore 'badConfig2' using instanceDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\badConfig2
   [junit4]   2> 2648551 T9805 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\badConfig2\'
   [junit4]   2> 2648570 T9805 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
   [junit4]   2> 2648576 T9805 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 2648576 T9805 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 2648581 T9805 oass.IndexSchema.readSchema [badConfig2] Schema name=tiny
   [junit4]   2> 2648584 T9805 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 2648584 T9805 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 2648586 T9805 oasc.SolrCore.<init> [badConfig2] Opening new SolrCore at C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\badConfig2\, dataDir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\badConfig2\data\
   [junit4]   2> 2648586 T9805 oasc.SolrCore.<init> JMX monitoring not detected for core: badConfig2
   [junit4]   2> 2648587 T9805 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\badConfig2\data\
   [junit4]   2> 2648587 T9805 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\badConfig2\data\index/
   [junit4]   2> 2648587 T9805 oasc.SolrCore.initIndex WARN [badConfig2] Solr index directory 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\badConfig2\data\index' doesn't exist. Creating new index...
   [junit4]   2> 2648588 T9805 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\badConfig2\data\index
   [junit4]   2> 2648588 T9805 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=43, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.5338438552709478]
   [junit4]   2> 2648588 T9805 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@3c2f3a57 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@56503283),segFN=segments_1,generation=1}
   [junit4]   2> 2648588 T9805 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 2648590 T9805 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 2648590 T9805 oasc.RequestHandlers.initHandlersFromConfig created /select: solr.SearchHandler
   [junit4]   2> 2648591 T9805 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 2648591 T9805 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 2648593 T9805 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 2648594 T9805 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 2648594 T9805 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 2648594 T9805 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=17, maxMergeAtOnceExplicit=30, maxMergedSegmentMB=92.9423828125, floorSegmentMB=1.287109375, forceMergeDeletesPctAllowed=2.6466250626050636, segmentsPerTier=35.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
   [junit4]   2> 2648594 T9805 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@3c2f3a57 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@56503283),segFN=segments_1,generation=1}
   [junit4]   2> 2648594 T9805 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 2648594 T9805 oass.SolrIndexSearcher.<init> Opening Searcher@26bd726d[badConfig2] main
   [junit4]   2> 2648596 T9839 oasc.SolrCore.registerSearcher [badConfig1] Registered new searcher Searcher@4d83802e[badConfig1] main{StandardDirectoryReader(segments_1:1:nrt)}
   [junit4]   2> 2648596 T9841 oasc.SolrCore.registerSearcher [badConfig2] Registered new searcher Searcher@26bd726d[badConfig2] main{StandardDirectoryReader(segments_1:1:nrt)}
   [junit4]   2> 2648596 T9805 oasc.SolrCores.putTransientCore Opening transient core badConfig2
   [junit4]   2> 2648597 T9805 oasc.CoreContainer.registerCore registering core: badConfig2
   [junit4]   2> 2648597 T9805 oasc.CoreContainer.create Creating SolrCore 'badSchema1' using instanceDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\badSchema1
   [junit4]   2> 2648597 T9805 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\badSchema1\'
   [junit4]   2> 2648616 T9805 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
   [junit4]   2> 2648628 T9805 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 2648630 T9805 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 2648633 T9805 oass.IndexSchema.readSchema [badSchema1] Schema name=tiny
   [junit4]   2> 2648636 T9805 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 2648636 T9805 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 2648637 T9805 oasc.SolrCore.<init> [badSchema1] Opening new SolrCore at C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\badSchema1\, dataDir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\badSchema1\data\
   [junit4]   2> 2648637 T9805 oasc.SolrCore.<init> JMX monitoring not detected for core: badSchema1
   [junit4]   2> 2648637 T9805 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\badSchema1\data\
   [junit4]   2> 2648637 T9805 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\badSchema1\data\index/
   [junit4]   2> 2648637 T9805 oasc.SolrCore.initIndex WARN [badSchema1] Solr index directory 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\badSchema1\data\index' doesn't exist. Creating new index...
   [junit4]   2> 2648638 T9805 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\badSchema1\data\index
   [junit4]   2> 2648639 T9805 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=39, maxMergeAtOnceExplicit=46, maxMergedSegmentMB=71.4384765625, floorSegmentMB=1.9462890625, forceMergeDeletesPctAllowed=0.7048446762804661, segmentsPerTier=39.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.816908068604344
   [junit4]   2> 2648640 T9805 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@7a35da3 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@3d3cd9d6),segFN=segments_1,generation=1}
   [junit4]   2> 2648640 T9805 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 2648641 T9805 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 2648641 T9805 oasc.RequestHandlers.initHandlersFromConfig created /select: solr.SearchHandler
   [junit4]   2> 2648642 T9805 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 2648642 T9805 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 2648643 T9805 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 2648645 T9805 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 2648645 T9805 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 2648645 T9805 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=29, maxMergeAtOnceExplicit=23, maxMergedSegmentMB=14.3291015625, floorSegmentMB=1.650390625, forceMergeDeletesPctAllowed=17.97748267509841, segmentsPerTier=26.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
   [junit4]   2> 2648646 T9805 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@7a35da3 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@3d3cd9d6),segFN=segments_1,generation=1}
   [junit4]   2> 2648646 T9805 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 2648646 T9805 oass.SolrIndexSearcher.<init> Opening Searcher@10daf675[badSchema1] main
   [junit4]   2> 2648647 T9805 oasc.SolrCores.putTransientCore Opening transient core badSchema1
   [junit4]   2> 2648647 T9805 oasc.CoreContainer.registerCore registering core: badSchema1
   [junit4]   2> 2648647 T9805 oasc.CoreContainer.create Creating SolrCore 'badSchema2' using instanceDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\badSchema2
   [junit4]   2> 2648647 T9805 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\badSchema2\'
   [junit4]   2> 2648666 T9805 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
   [junit4]   2> 2648671 T9805 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 2648671 T9805 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 2648674 T9805 oass.IndexSchema.readSchema [badSchema2] Schema name=tiny
   [junit4]   2> 2648677 T9805 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 2648677 T9805 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 2648678 T9805 oasc.SolrCore.<init> [badSchema2] Opening new SolrCore at C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\badSchema2\, dataDir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\badSchema2\data\
   [junit4]   2> 2648678 T9805 oasc.SolrCore.<init> JMX monitoring not detected for core: badSchema2
   [junit4]   2> 2648679 T9843 oasc.SolrCore.registerSearcher [badSchema1] Registered new searcher Searcher@10daf675[badSchema1] main{StandardDirectoryReader(segments_1:1:nrt)}
   [junit4]   2> 2648682 T9805 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\badSchema2\data\
   [junit4]   2> 2648682 T9805 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\badSchema2\data\index/
   [junit4]   2> 2648682 T9805 oasc.SolrCore.initIndex WARN [badSchema2] Solr index directory 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\badSchema2\data\index' doesn't exist. Creating new index...
   [junit4]   2> 2648683 T9805 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\badSchema2\data\index
   [junit4]   2> 2648683 T9805 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=44, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
   [junit4]   2> 2648683 T9805 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@3ab23bdf lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@67ff33d1),segFN=segments_1,generation=1}
   [junit4]   2> 2648683 T9805 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 2648685 T9805 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 2648685 T9805 oasc.RequestHandlers.initHandlersFromConfig created /select: solr.SearchHandler
   [junit4]   2> 2648686 T9805 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 2648686 T9805 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 2648687 T9805 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 2648698 T9805 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 2648698 T9805 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 2648698 T9805 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=47, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.707704457859236]
   [junit4]   2> 2648699 T9805 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@3ab23bdf lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@67ff33d1),segFN=segments_1,generation=1}
   [junit4]   2> 2648699 T9805 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 2648699 T9805 oass.SolrIndexSearcher.<init> Opening Searcher@10731f92[badSchema2] main
   [junit4]   2> 2648700 T9805 oasc.SolrCores.putTransientCore Opening transient core badSchema2
   [junit4]   2> 2648700 T9805 oasc.CoreContainer.registerCore registering core: badSchema2
   [junit4]   2> 2648701 T9845 oasc.SolrCore.registerSearcher [badSchema2] Registered new searcher Searcher@10731f92[badSchema2] main{StandardDirectoryReader(segments_1:1:nrt)}
   [junit4]   2> 2648701 T9805 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 2648701 T9805 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@3e7bdc38 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@294cd809),segFN=segments_2,generation=2}
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@3e7bdc38 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@294cd809),segFN=segments_3,generation=3}
   [junit4]   2> 2648703 T9805 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
   [junit4]   2> 2648703 T9805 oass.SolrIndexSearcher.<init> Opening Searcher@2214e552[core1] main
   [junit4]   2> 2648703 T9805 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 2648704 T9837 oasc.SolrCore.registerSearcher [core1] Registered new searcher Searcher@2214e552[core1] main{StandardDirectoryReader(segments_3:7:nrt _1(5.0):c7)}
   [junit4]   2> ASYNC  NEW_CORE C4367 name=core1 org.apache.solr.core.SolrCore@75de95c1
   [junit4]   2> 2648705 T9805 C4367 oasc.SolrCore.execute [collection1] webapp=null path=null params={q={!prefix+f%3Dv_t}hel&wt=xml} hits=2 status=0 QTime=1 
   [junit4]   2> 2648705 T9805 C4367 oasc.SolrCore.execute [collection1] webapp=null path=null params={q={!raw+f%3Dv_t}hello&wt=xml} hits=2 status=0 QTime=0 
   [junit4]   2> 2648706 T9805 C4367 oasc.SolrCore.execute [collection1] webapp=null path=null params={q={!raw+f%3Dv_t}Hello&wt=xml} hits=0 status=0 QTime=1 
   [junit4]   2> 2648706 T9805 C4367 oasc.SolrCore.execute [collection1] webapp=null path=null params={q={!raw+f%3Dv_f}1.5&wt=xml} hits=0 status=0 QTime=0 
   [junit4]   2> 2648706 T9805 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 2648706 T9805 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@6156d19f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1aa126c),segFN=segments_1,generation=1}
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@6156d19f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1aa126c),segFN=segments_2,generation=2}
   [junit4]   2> 2648706 T9805 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
   [junit4]   2> 2648706 T9805 oass.SolrIndexSearcher.<init> Opening Searcher@253c908[badConfig1] main
   [junit4]   2> 2648708 T9805 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 2648708 T9839 oasc.SolrCore.registerSearcher [badConfig1] Registered new searcher Searcher@253c908[badConfig1] main{StandardDirectoryReader(segments_2:3:nrt _0(5.0):c7)}
   [junit4]   2> ASYNC  NEW_CORE C4368 name=badConfig1 org.apache.solr.core.SolrCore@79a88845
   [junit4]   2> 2648708 T9805 C4368 oasc.SolrCore.execute [collection1] webapp=null path=null params={q={!prefix+f%3Dv_t}hel&wt=xml} hits=2 status=0 QTime=0 
   [junit4]   2> 2648709 T9805 C4368 oasc.SolrCore.execute [collection1] webapp=null path=null params={q={!raw+f%3Dv_t}hello&wt=xml} hits=2 status=0 QTime=1 
   [junit4]   2> 2648709 T9805 C4368 oasc.SolrCore.execute [collection1] webapp=null path=null params={q={!raw+f%3Dv_t}Hello&wt=xml} hits=0 status=0 QTime=0 
   [junit4]   2> 2648710 T9805 C4368 oasc.SolrCore.execute [collection1] webapp=null path=null params={q={!raw+f%3Dv_f}1.5&wt=xml} hits=0 status=0 QTime=1 
   [junit4]   2> 2648710 T9805 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 2648710 T9805 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@7a35da3 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@3d3cd9d6),segFN=segments_1,generation=1}
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@7a35da3 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@3d3cd9d6),segFN=segments_2,generation=2}
   [junit4]   2> 2648712 T9805 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
   [junit4]   2> 2648712 T9805 oass.SolrIndexSearcher.<init> Opening Searcher@6ff3b146[badSchema1] main
   [junit4]   2> 2648712 T9805 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 2648713 T9843 oasc.SolrCore.registerSearcher [badSchema1] Registered new searcher Searcher@6ff3b146[badSchema1] main{StandardDirectoryReader(segments_2:3:nrt _0(5.0):c7)}
   [junit4]   2> ASYNC  NEW_CORE C4369 name=badSchema1 org.apache.solr.core.SolrCore@566c7bfe
   [junit4]   2> 2648713 T9805 C4369 oasc.SolrCore.execute [collection1] webapp=null path=null params={q={!prefix+f%3Dv_t}hel&wt=xml} hits=2 status=0 QTime=0 
   [junit4]   2> 2648714 T9805 C4369 oasc.SolrCore.execute [collection1] webapp=null path=null params={q={!raw+f%3Dv_t}hello&wt=xml} hits=2 status=0 QTime=1 
   [junit4]   2> 2648714 T9805 C4369 oasc.SolrCore.execute [collection1] webapp=null path=null params={q={!raw+f%3Dv_t}Hello&wt=xml} hits=0 status=0 QTime=0 
   [junit4]   2> 2648715 T9805 C4369 oasc.SolrCore.execute [collection1] webapp=null path=null params={q={!raw+f%3Dv_f}1.5&wt=xml} hits=0 status=0 QTime=1 
   [junit4]   2> 2648715 T9805 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=66674091
   [junit4]   2> 2648715 T9805 oasc.SolrCore.close [core2]  CLOSING SolrCore org.apache.solr.core.SolrCore@3fac45e9
   [junit4]   2> 2648715 T9805 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
   [junit4]   2> 2648715 T9805 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
   [junit4]   2> 2648715 T9805 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
   [junit4]   2> 2648715 T9805 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
   [junit4]   2> 2648718 T9805 oasc.SolrCore.closeSearcher [core2] Closing main searcher on request.
   [junit4]   2> 2648718 T9805 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
   [junit4]   2> 2648718 T9805 oasc.CachingDirectoryFactory.closeCacheValue looking to close C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\core2\data\index [CachedDir<<refCount=0;path=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\TestLazyCores\core2\data\index;done=false>>]
   [junit4]   2> 264871

[...truncated too long message...]

ldn't be terminated:
   [junit4]   2> 	   1) Thread[id=9805, name=SUITE-TestLazyCores-seed#[1E468F4F611387A5]-worker, state=TIMED_WAITING, group=TGRP-TestLazyCores]
   [junit4]   2> 	        at java.lang.Thread.sleep(Native Method)
   [junit4]   2> 	        at org.apache.solr.SolrTestCaseJ4.endTrackingSearchers(SolrTestCaseJ4.java:399)
   [junit4]   2> 	        at org.apache.solr.SolrTestCaseJ4.afterClass(SolrTestCaseJ4.java:174)
   [junit4]   2> 	        at sun.reflect.GeneratedMethodAccessor51.invoke(Unknown Source)
   [junit4]   2> 	        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2> 	        at java.lang.reflect.Method.invoke(Method.java:483)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:700)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
   [junit4]   2> 	        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2> 	        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
   [junit4]   2> 	        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2> 	        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
   [junit4]   2> 	        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
   [junit4]   2> 	        at java.lang.Thread.run(Thread.java:744)
   [junit4]   2> NOTE: test params are: codec=Lucene45, sim=DefaultSimilarity, locale=vi, timezone=Antarctica/McMurdo
   [junit4]   2> NOTE: Windows 7 6.1 amd64/Oracle Corporation 1.8.0 (64-bit)/cpus=2,threads=2,free=328515352,total=506900480
   [junit4]   2> NOTE: All tests run in this JVM: [ChangedSchemaMergeTest, TestPostingsSolrHighlighter, DistributedTermsComponentTest, DistributedQueryElevationComponentTest, CurrencyFieldOpenExchangeTest, TestComponentsName, SignatureUpdateProcessorFactoryTest, TestNonDefinedSimilarityFactory, TestStressRecovery, PathHierarchyTokenizerFactoryTest, TestSweetSpotSimilarityFactory, DOMUtilTest, SliceStateUpdateTest, TestSolrXmlPersistor, TestRecovery, HdfsUnloadDistributedZkTest, SortByFunctionTest, TestReloadAndDeleteDocs, TestStressLucene, StatelessScriptUpdateProcessorFactoryTest, TestFieldCollectionResource, TestLFUCache, TestSearchPerf, InfoHandlerTest, ShardSplitTest, TestFuzzyAnalyzedSuggestions, CopyFieldTest, TestFaceting, TermVectorComponentDistributedTest, TestQueryUtils, TestRemoteStreaming, ModifyConfFileTest, TestSchemaNameResource, TestGroupingSearch, TestManagedSchema, UnloadDistributedZkTest, CurrencyFieldXmlFileTest, TestManagedSchemaFieldResource, TestIBSimilarityFactory, TestJmxMonitoredMap, TestIndexingPerformance, TestBinaryResponseWriter, DirectUpdateHandlerOptimizeTest, DocumentBuilderTest, ShowFileRequestHandlerTest, BasicFunctionalityTest, TestRecoveryHdfs, XsltUpdateRequestHandlerTest, EchoParamsTest, ConvertedLegacyTest, TestRTGBase, TestDocBasedVersionConstraints, TestSort, HdfsChaosMonkeySafeLeaderTest, SolrCoreTest, TestFieldResource, UniqFieldsUpdateProcessorFactoryTest, DistributedSuggestComponentTest, TestSolrQueryParserResource, SearchHandlerTest, DistributedSpellCheckComponentTest, TestDFRSimilarityFactory, TestFastOutputStream, TestSchemaResource, UpdateRequestProcessorFactoryTest, DateMathParserTest, SpellCheckComponentTest, FileBasedSpellCheckerTest, TestValueSourceCache, SimplePostToolTest, LukeRequestHandlerTest, SuggestComponentTest, RequiredFieldsTest, TestQueryTypes, TestWriterPerf, TestOverriddenPrefixQueryForCustomFieldType, StressHdfsTest, XmlUpdateRequestHandlerTest, TestFreeTextSuggestions, ScriptEngineTest, TestMaxScoreQueryParser, PrimUtilsTest, SolrCoreCheckLockOnStartupTest, TestSimpleQParserPlugin, HdfsLockFactoryTest, TestNonNRTOpen, MultiTermTest, TermVectorComponentTest, TestExtendedDismaxParser, SOLR749Test, TestAnalyzeInfixSuggestions, CoreContainerCoreInitFailuresTest, TestSolrQueryParser, AnalysisAfterCoreReloadTest, ZkSolrClientTest, TestSolrIndexConfig, TestSolrCoreProperties, ReturnFieldsTest, TestZkChroot, BadComponentTest, TestCloudManagedSchemaAddField, SyncSliceTest, OutputWriterTest, TestFieldTypeResource, PluginInfoTest, CustomCollectionTest, TestCSVLoader, TestCodecSupport, TestQuerySenderNoQuery, PreAnalyzedFieldTest, FileUtilsTest, DirectSolrConnectionTest, HdfsSyncSliceTest, ClusterStateTest, FieldAnalysisRequestHandlerTest, TestSolrDeletionPolicy1, LeaderElectionIntegrationTest, TestDistributedGrouping, TestHighlightDedupGrouping, SolrIndexConfigTest, ChaosMonkeyNothingIsSafeTest, FieldFacetExtrasTest, TermsComponentTest, ChaosMonkeySafeLeaderTest, OverseerCollectionProcessorTest, TestAnalyzedSuggestions, StandardRequestHandlerTest, TestStressUserVersions, BadIndexSchemaTest, DefaultValueUpdateProcessorTest, TestStressVersions, ResponseLogComponentTest, TestPseudoReturnFields, LoggingHandlerTest, DeleteReplicaTest, TestDynamicFieldResource, TestCharFilters, MigrateRouteKeyTest, TestBinaryField, RecoveryZkTest, TestDocumentBuilder, CoreAdminHandlerTest, BasicZkTest, JSONWriterTest, QueryFacetTest, TestQuerySenderListener, TestSolrXMLSerializer, TestCoreDiscovery, NoFacetTest, TestSuggestSpellingConverter, DistanceFunctionTest, TestRangeQuery, BinaryUpdateRequestHandlerTest, OverseerRolesTest, TestIndexSearcher, RAMDirectoryFactoryTest, AlternateDirectoryTest, SliceStateTest, TestOmitPositions, TestFoldingMultitermQuery, TestXIncludeConfig, ConnectionManagerTest, TestModifyConfFiles, DistributedDebugComponentTest, ExpressionTest, SpatialFilterTest, TestCopyFieldCollectionResource, TestPerFieldSimilarity, TestSchemaVersionResource, TestInitQParser, TestCursorMarkWithoutUniqueKey, TestSolr4Spatial, DebugComponentTest, TestReplicationHandler, FullSolrCloudDistribCmdsTest, BasicDistributedZk2Test, TestStressReorder, NumericFieldsTest, HardAutoCommitTest, TestSolrJ, BasicDistributedZkTest, CollectionsAPIDistributedZkTest, OpenCloseCoreStressTest, OverseerTest, ShardRoutingTest, ClusterStateUpdateTest, TestRandomFaceting, LeaderElectionTest, TestRandomDVFaceting, ZkCLITest, ShardRoutingCustomTest, TestDistributedSearch, AutoCommitTest, TestHashPartitioner, ZkControllerTest, TestRealTimeGet, TestJoin, TestMultiCoreConfBootstrap, TestReload, TestCoreContainer, SimpleFacetsTest, StatsComponentTest, SolrCmdDistributorTest, QueryElevationComponentTest, PeerSyncTest, TestFiltering, TestFunctionQuery, TestLazyCores]
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestLazyCores -Dtests.seed=1E468F4F611387A5 -Dtests.slow=true -Dtests.locale=vi -Dtests.timezone=Antarctica/McMurdo -Dtests.file.encoding=Cp1252
   [junit4] ERROR   0.00s | TestLazyCores (suite) <<<
   [junit4]    > Throwable #1: java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([1E468F4F611387A5]:0)
   [junit4] Completed in 7257.19s, 8 tests, 2 errors <<< FAILURES!

[...truncated 484 lines...]
BUILD FAILED
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\build.xml:465: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\build.xml:445: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\build.xml:39: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\extra-targets.xml:37: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build.xml:189: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\common-build.xml:490: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\common-build.xml:1274: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\common-build.xml:907: There were test failures: 373 suites, 1606 tests, 1 suite-level error, 1 error, 693 ignored (9 assumptions)

Total time: 192 minutes 50 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 64bit/jdk1.8.0-fcs-b129 -XX:-UseCompressedOops -XX:+UseConcMarkSweepGC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-trunk-Windows (32bit/jdk1.7.0_60-ea-b04) - Build # 3819 - Still Failing!

Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Windows/3819/
Java: 32bit/jdk1.7.0_60-ea-b04 -client -XX:+UseSerialGC

All tests passed

Build Log:
[...truncated 50551 lines...]
BUILD FAILED
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\build.xml:465: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\build.xml:404: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\extra-targets.xml:87: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\extra-targets.xml:187: Source checkout is dirty after running tests!!! Offending files:
* ./solr/licenses/jcl-over-slf4j-1.6.6.jar.sha1
* ./solr/licenses/jul-to-slf4j-1.6.6.jar.sha1
* ./solr/licenses/slf4j-api-1.6.6.jar.sha1
* ./solr/licenses/slf4j-log4j12-1.6.6.jar.sha1

Total time: 117 minutes 53 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 32bit/jdk1.7.0_60-ea-b04 -client -XX:+UseSerialGC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



Re: [JENKINS] Lucene-Solr-trunk-Windows (64bit/jdk1.7.0_60-ea-b04) - Build # 3818 - Still Failing!

Posted by Mark Miller <ma...@gmail.com>.
I have not seen this before. If it pops up again, I’ll raise the zk timeout being used.

- Mark

http://about.me/markrmiller

On Feb 26, 2014, at 2:17 PM, Policeman Jenkins Server <je...@thetaphi.de> wrote:

> Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Windows/3818/
> Java: 64bit/jdk1.7.0_60-ea-b04 -XX:+UseCompressedOops -XX:+UseSerialGC
> 
> 1 tests failed.
> REGRESSION:  org.apache.solr.cloud.ChaosMonkeySafeLeaderTest.testDistribSearch
> 
> Error Message:
> KeeperErrorCode = Session expired for /live_nodes
> 
> Stack Trace:
> org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /live_nodes
> 	at __randomizedtesting.SeedInfo.seed([DF1F466614BE0A92:5EF9C87E63E16AAE]:0)
> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
> 	at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
> 	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:257)
> 	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:254)
> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
> 	at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:254)
> 	at org.apache.solr.common.cloud.ZkStateReader.updateClusterState(ZkStateReader.java:404)
> 	at org.apache.solr.common.cloud.ZkStateReader.updateClusterState(ZkStateReader.java:230)
> 	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:144)
> 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.waitForRecoveriesToFinish(AbstractFullDistribZkTestBase.java:730)
> 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.waitForThingsToLevelOut(AbstractFullDistribZkTestBase.java:1602)
> 	at org.apache.solr.cloud.ChaosMonkeySafeLeaderTest.doTest(ChaosMonkeySafeLeaderTest.java:148)
> 	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:870)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 	at java.lang.reflect.Method.invoke(Method.java:606)
> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
> 	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
> 	at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
> 	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
> 	at java.lang.Thread.run(Thread.java:744)
> 
> 
> 
> 
> Build Log:
> [...truncated 19360 lines...]
>   [junit4] Suite: org.apache.solr.cloud.ChaosMonkeySafeLeaderTest
>   [junit4]   2> 699895 T1723 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (true) and clientAuth (false)
>   [junit4]   2> 699895 T1723 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /m_bk/
>   [junit4]   2> 699898 T1723 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
>   [junit4]   2> Creating dataDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\solrtest-ChaosMonkeySafeLeaderTest-1393439503084
>   [junit4]   2> 699899 T1723 oasc.ZkTestServer.run STARTING ZK TEST SERVER
>   [junit4]   2> 699900 T1724 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
>   [junit4]   2> 699973 T1723 oasc.ZkTestServer.run start zk server on port:55587
>   [junit4]   2> 699973 T1723 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
>   [junit4]   2> 699980 T1730 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5cdc87da name:ZooKeeperConnection Watcher:127.0.0.1:55587 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
>   [junit4]   2> 699980 T1723 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
>   [junit4]   2> 699980 T1723 oascc.SolrZkClient.makePath makePath: /solr
>   [junit4]   2> 699985 T1725 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x1446f7772f10000, likely client has closed socket
>   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:220)
>   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
>   [junit4]   2> 	at java.lang.Thread.run(Thread.java:744)
>   [junit4]   2> 
>   [junit4]   2> 699986 T1723 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
>   [junit4]   2> 699988 T1732 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3583c568 name:ZooKeeperConnection Watcher:127.0.0.1:55587/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
>   [junit4]   2> 699988 T1723 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
>   [junit4]   2> 699988 T1723 oascc.SolrZkClient.makePath makePath: /collections/collection1
>   [junit4]   2> 699993 T1723 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
>   [junit4]   2> 699996 T1723 oascc.SolrZkClient.makePath makePath: /collections/control_collection
>   [junit4]   2> 700000 T1723 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
>   [junit4]   2> 700003 T1723 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
>   [junit4]   2> 700003 T1723 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
>   [junit4]   2> 700009 T1723 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\schema15.xml to /configs/conf1/schema.xml
>   [junit4]   2> 700010 T1723 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
>   [junit4]   2> 700013 T1723 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
>   [junit4]   2> 700013 T1723 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
>   [junit4]   2> 700017 T1723 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\stopwords.txt to /configs/conf1/stopwords.txt
>   [junit4]   2> 700017 T1723 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
>   [junit4]   2> 700021 T1723 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\protwords.txt to /configs/conf1/protwords.txt
>   [junit4]   2> 700021 T1723 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
>   [junit4]   2> 700024 T1723 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\currency.xml to /configs/conf1/currency.xml
>   [junit4]   2> 700024 T1723 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
>   [junit4]   2> 700028 T1723 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\enumsConfig.xml to /configs/conf1/enumsConfig.xml
>   [junit4]   2> 700028 T1723 oascc.SolrZkClient.makePath makePath: /configs/conf1/enumsConfig.xml
>   [junit4]   2> 700031 T1723 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
>   [junit4]   2> 700031 T1723 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
>   [junit4]   2> 700035 T1723 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
>   [junit4]   2> 700036 T1723 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
>   [junit4]   2> 700039 T1723 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\old_synonyms.txt to /configs/conf1/old_synonyms.txt
>   [junit4]   2> 700039 T1723 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
>   [junit4]   2> 700041 T1723 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\synonyms.txt to /configs/conf1/synonyms.txt
>   [junit4]   2> 700043 T1723 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
>   [junit4]   2> 700048 T1723 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
>   [junit4]   2> 700051 T1734 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7b314dd8 name:ZooKeeperConnection Watcher:127.0.0.1:55587/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
>   [junit4]   2> 700051 T1723 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
>   [junit4]   2> 700054 T1725 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x1446f7772f10002, likely client has closed socket
>   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:220)
>   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
>   [junit4]   2> 	at java.lang.Thread.run(Thread.java:744)
>   [junit4]   2> 
>   [junit4]   2> 701313 T1723 oejs.Server.doStart jetty-8.1.10.v20130312
>   [junit4]   2> 701319 T1723 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
>   [junit4]   2> 701323 T1723 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:55597
>   [junit4]   2> 701326 T1723 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
>   [junit4]   2> 701326 T1723 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
>   [junit4]   2> 701326 T1723 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-controljetty-1393439503239
>   [junit4]   2> 701326 T1723 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-controljetty-1393439503239\'
>   [junit4]   2> 701355 T1723 oasc.ConfigSolr.fromFile Loading container configuration from C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-controljetty-1393439503239\solr.xml
>   [junit4]   2> 701393 T1723 oasc.CoreContainer.<init> New CoreContainer 649471709
>   [junit4]   2> 701393 T1723 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-controljetty-1393439503239\]
>   [junit4]   2> 701395 T1723 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
>   [junit4]   2> 701395 T1723 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
>   [junit4]   2> 701395 T1723 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
>   [junit4]   2> 701395 T1723 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
>   [junit4]   2> 701395 T1723 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
>   [junit4]   2> 701395 T1723 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
>   [junit4]   2> 701395 T1723 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
>   [junit4]   2> 701395 T1723 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
>   [junit4]   2> 701395 T1723 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
>   [junit4]   2> 701403 T1723 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
>   [junit4]   2> 701403 T1723 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
>   [junit4]   2> 701403 T1723 oasc.CoreContainer.load Host Name: 127.0.0.1
>   [junit4]   2> 701403 T1723 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:55587/solr
>   [junit4]   2> 701403 T1723 oasc.ZkController.checkChrootPath zkHost includes chroot
>   [junit4]   2> 701405 T1723 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
>   [junit4]   2> 701409 T1745 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1734851a name:ZooKeeperConnection Watcher:127.0.0.1:55587 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
>   [junit4]   2> 701409 T1723 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
>   [junit4]   2> 701412 T1725 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x1446f7772f10003, likely client has closed socket
>   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:220)
>   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
>   [junit4]   2> 	at java.lang.Thread.run(Thread.java:744)
>   [junit4]   2> 
>   [junit4]   2> 701413 T1723 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
>   [junit4]   2> 701415 T1747 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5a9ffb81 name:ZooKeeperConnection Watcher:127.0.0.1:55587/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
>   [junit4]   2> 701415 T1723 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
>   [junit4]   2> 701418 T1723 oascc.SolrZkClient.makePath makePath: /overseer/queue
>   [junit4]   2> 701424 T1723 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
>   [junit4]   2> 701431 T1723 oascc.SolrZkClient.makePath makePath: /live_nodes
>   [junit4]   2> 701434 T1723 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:55597_m_bk
>   [junit4]   2> 701435 T1723 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:55597_m_bk
>   [junit4]   2> 701442 T1723 oascc.SolrZkClient.makePath makePath: /overseer_elect
>   [junit4]   2> 701446 T1723 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
>   [junit4]   2> 701453 T1723 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:55597_m_bk
>   [junit4]   2> 701453 T1723 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
>   [junit4]   2> 701457 T1723 oasc.Overseer.start Overseer (id=91320451274440708-127.0.0.1:55597_m_bk-n_0000000000) starting
>   [junit4]   2> 701463 T1723 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
>   [junit4]   2> 701471 T1749 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
>   [junit4]   2> 701472 T1723 oascc.SolrZkClient.makePath makePath: /clusterstate.json
>   [junit4]   2> 701473 T1749 oasc.OverseerCollectionProcessor.prioritizeOverseerNodes prioritizing overseer nodes
>   [junit4]   2> 701477 T1723 oascc.SolrZkClient.makePath makePath: /aliases.json
>   [junit4]   2> 701480 T1723 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
>   [junit4]   2> 701484 T1748 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
>   [junit4]   2> 701489 T1750 oasc.ZkController.publish publishing core=collection1 state=down
>   [junit4]   2> 701489 T1750 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
>   [junit4]   2> 701491 T1750 oasc.ZkController.waitForCoreNodeName look for our core node name
>   [junit4]   2> 701491 T1747 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
>   [junit4]   2> 701494 T1748 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
>   [junit4]   2> 701495 T1748 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
>   [junit4]   2> 	  "operation":"state",
>   [junit4]   2> 	  "state":"down",
>   [junit4]   2> 	  "base_url":"https://127.0.0.1:55597/m_bk",
>   [junit4]   2> 	  "core":"collection1",
>   [junit4]   2> 	  "roles":null,
>   [junit4]   2> 	  "node_name":"127.0.0.1:55597_m_bk",
>   [junit4]   2> 	  "shard":null,
>   [junit4]   2> 	  "collection":"control_collection",
>   [junit4]   2> 	  "numShards":"1",
>   [junit4]   2> 	  "core_node_name":null}
>   [junit4]   2> 701495 T1748 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with shards [shard1]
>   [junit4]   2> 701495 T1748 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
>   [junit4]   2> 701499 T1747 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
>   [junit4]   2> 701499 T1747 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
>   [junit4]   2> 702289 T1750 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
>   [junit4]   2> 702289 T1750 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-controljetty-1393439503239\collection1
>   [junit4]   2> 702289 T1750 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
>   [junit4]   2> 702290 T1750 oasc.ZkController.createCollectionZkNode Collection zkNode exists
>   [junit4]   2> 702290 T1750 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
>   [junit4]   2> 702292 T1750 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
>   [junit4]   2> 702292 T1750 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-controljetty-1393439503239\collection1\'
>   [junit4]   2> 702296 T1750 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/C:/Users/JenkinsSlave/workspace/Lucene-Solr-trunk-Windows/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-controljetty-1393439503239/collection1/lib/.svn/' to classloader
>   [junit4]   2> 702297 T1750 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/C:/Users/JenkinsSlave/workspace/Lucene-Solr-trunk-Windows/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-controljetty-1393439503239/collection1/lib/classes/' to classloader
>   [junit4]   2> 702299 T1750 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/C:/Users/JenkinsSlave/workspace/Lucene-Solr-trunk-Windows/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-controljetty-1393439503239/collection1/lib/README' to classloader
>   [junit4]   2> 702340 T1750 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
>   [junit4]   2> 702372 T1750 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
>   [junit4]   2> 702374 T1750 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
>   [junit4]   2> 702381 T1750 oass.IndexSchema.readSchema [collection1] Schema name=test
>   [junit4]   2> 702645 T1750 oass.IndexSchema.readSchema default search field in schema is text
>   [junit4]   2> 702646 T1750 oass.IndexSchema.readSchema unique key field: id
>   [junit4]   2> 702647 T1750 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
>   [junit4]   2> 702652 T1750 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
>   [junit4]   2> 702683 T1750 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
>   [junit4]   2> 702684 T1750 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-controljetty-1393439503239\collection1\, dataDir=.\org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393439503084/control/data\
>   [junit4]   2> 702684 T1750 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@79d7c4fb
>   [junit4]   2> 702684 T1750 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393439503084\control\data
>   [junit4]   2> 702685 T1750 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=.\org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393439503084/control/data\index/
>   [junit4]   2> 702686 T1750 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '.\org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393439503084\control\data\index' doesn't exist. Creating new index...
>   [junit4]   2> 702686 T1750 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393439503084\control\data\index
>   [junit4]   2> 702686 T1750 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=39, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
>   [junit4]   2> 702691 T1750 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
>   [junit4]   2> 		commit{dir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393439503084\control\data\index,segFN=segments_1,generation=1}
>   [junit4]   2> 702691 T1750 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
>   [junit4]   2> 702694 T1750 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
>   [junit4]   2> 702694 T1750 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
>   [junit4]   2> 702695 T1750 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
>   [junit4]   2> 702695 T1750 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
>   [junit4]   2> 702695 T1750 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
>   [junit4]   2> 702695 T1750 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
>   [junit4]   2> 702695 T1750 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
>   [junit4]   2> 702695 T1750 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
>   [junit4]   2> 702695 T1750 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
>   [junit4]   2> 702696 T1750 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
>   [junit4]   2> 702696 T1750 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
>   [junit4]   2> 702696 T1750 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
>   [junit4]   2> 702696 T1750 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
>   [junit4]   2> 702697 T1750 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
>   [junit4]   2> 702697 T1750 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
>   [junit4]   2> 702697 T1750 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
>   [junit4]   2> 702698 T1750 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
>   [junit4]   2> 702704 T1750 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
>   [junit4]   2> 702707 T1750 oasu.CommitTracker.<init> Hard AutoCommit: disabled
>   [junit4]   2> 702707 T1750 oasu.CommitTracker.<init> Soft AutoCommit: disabled
>   [junit4]   2> 702707 T1750 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=1854487036, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
>   [junit4]   2> 702709 T1750 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
>   [junit4]   2> 		commit{dir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393439503084\control\data\index,segFN=segments_1,generation=1}
>   [junit4]   2> 702709 T1750 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
>   [junit4]   2> 702709 T1750 oass.SolrIndexSearcher.<init> Opening Searcher@3f73a3dd[collection1] main
>   [junit4]   2> 702712 T1751 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3f73a3dd[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
>   [junit4]   2> 702713 T1750 oasc.CoreContainer.registerCore registering core: collection1
>   [junit4]   2> 702714 T1723 oass.SolrDispatchFilter.init user.dir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0
>   [junit4]   2> 702714 T1723 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
>   [junit4]   2> 702714 T1754 oasc.ZkController.register Register replica - core:collection1 address:https://127.0.0.1:55597/m_bk collection:control_collection shard:shard1
>   [junit4]   2> 702716 T1754 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
>   [junit4]   2> 702723 T1723 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
>   [junit4]   2> 702726 T1756 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@21dc2fac name:ZooKeeperConnection Watcher:127.0.0.1:55587/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
>   [junit4]   2> 702726 T1723 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
>   [junit4]   2> 702727 T1754 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
>   [junit4]   2> 702730 T1723 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
>   [junit4]   2> 702731 T1747 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
>   [junit4]   2> 702732 T1754 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
>   [junit4]   2> 702732 T1754 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
>   [junit4]   2> ASYNC  NEW_CORE C1250 name=collection1 org.apache.solr.core.SolrCore@430e225d url=https://127.0.0.1:55597/m_bk/collection1 node=127.0.0.1:55597_m_bk C1250_STATE=coll:control_collection core:collection1 props:{state=down, base_url=https://127.0.0.1:55597/m_bk, core=collection1, node_name=127.0.0.1:55597_m_bk}
>   [junit4]   2> 702732 T1754 C1250 P55597 oasc.SyncStrategy.sync Sync replicas to https://127.0.0.1:55597/m_bk/collection1/
>   [junit4]   2> 702732 T1754 C1250 P55597 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
>   [junit4]   2> 702733 T1754 C1250 P55597 oasc.SyncStrategy.syncToMe https://127.0.0.1:55597/m_bk/collection1/ has no replicas
>   [junit4]   2> 702733 T1754 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: https://127.0.0.1:55597/m_bk/collection1/ shard1
>   [junit4]   2> 702733 T1754 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
>   [junit4]   2> 702734 T1723 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false
>   [junit4]   2> 702734 T1748 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
>   [junit4]   2> 702741 T1747 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
>   [junit4]   2> 702743 T1747 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
>   [junit4]   2> 702749 T1747 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
>   [junit4]   2> 702832 T1756 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
>   [junit4]   2> 702833 T1747 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
>   [junit4]   2> 702865 T1754 oasc.ZkController.register We are https://127.0.0.1:55597/m_bk/collection1/ and leader is https://127.0.0.1:55597/m_bk/collection1/
>   [junit4]   2> 702865 T1754 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:55597/m_bk
>   [junit4]   2> 702865 T1754 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
>   [junit4]   2> 702865 T1754 oasc.ZkController.publish publishing core=collection1 state=active
>   [junit4]   2> 702865 T1754 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
>   [junit4]   2> 702867 T1747 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
>   [junit4]   2> 702867 T1747 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
>   [junit4]   2> 702867 T1747 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
>   [junit4]   2> 702869 T1754 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
>   [junit4]   2> 702871 T1748 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
>   [junit4]   2> 702871 T1748 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
>   [junit4]   2> 	  "operation":"state",
>   [junit4]   2> 	  "state":"active",
>   [junit4]   2> 	  "base_url":"https://127.0.0.1:55597/m_bk",
>   [junit4]   2> 	  "core":"collection1",
>   [junit4]   2> 	  "roles":null,
>   [junit4]   2> 	  "node_name":"127.0.0.1:55597_m_bk",
>   [junit4]   2> 	  "shard":"shard1",
>   [junit4]   2> 	  "collection":"control_collection",
>   [junit4]   2> 	  "numShards":"3",
>   [junit4]   2> 	  "core_node_name":"core_node1"}
>   [junit4]   2> 702875 T1747 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
>   [junit4]   2> 702959 T1747 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
>   [junit4]   2> 702959 T1756 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
>   [junit4]   2> 704071 T1723 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 1
>   [junit4]   2> 704073 T1723 oejs.Server.doStart jetty-8.1.10.v20130312
>   [junit4]   2> 704080 T1723 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
>   [junit4]   2> 704081 T1723 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:55609
>   [junit4]   2> 704084 T1723 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
>   [junit4]   2> 704084 T1723 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
>   [junit4]   2> 704084 T1723 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty1-1393439505920
>   [junit4]   2> 704084 T1723 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty1-1393439505920\'
>   [junit4]   2> 704109 T1723 oasc.ConfigSolr.fromFile Loading container configuration from C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty1-1393439505920\solr.xml
>   [junit4]   2> 704152 T1723 oasc.CoreContainer.<init> New CoreContainer 701776337
>   [junit4]   2> 704153 T1723 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty1-1393439505920\]
>   [junit4]   2> 704154 T1723 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
>   [junit4]   2> 704154 T1723 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
>   [junit4]   2> 704154 T1723 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
>   [junit4]   2> 704154 T1723 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
>   [junit4]   2> 704155 T1723 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
>   [junit4]   2> 704155 T1723 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
>   [junit4]   2> 704155 T1723 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
>   [junit4]   2> 704155 T1723 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
>   [junit4]   2> 704155 T1723 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
>   [junit4]   2> 704162 T1723 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
>   [junit4]   2> 704162 T1723 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
>   [junit4]   2> 704163 T1723 oasc.CoreContainer.load Host Name: 127.0.0.1
>   [junit4]   2> 704163 T1723 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:55587/solr
>   [junit4]   2> 704163 T1723 oasc.ZkController.checkChrootPath zkHost includes chroot
>   [junit4]   2> 704165 T1723 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
>   [junit4]   2> 704187 T1767 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2c1fd552 name:ZooKeeperConnection Watcher:127.0.0.1:55587 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
>   [junit4]   2> 704187 T1723 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
>   [junit4]   2> 704208 T1725 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x1446f7772f10006, likely client has closed socket
>   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:220)
>   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
>   [junit4]   2> 	at java.lang.Thread.run(Thread.java:744)
>   [junit4]   2> 
>   [junit4]   2> 704212 T1723 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
>   [junit4]   2> 704214 T1769 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@b4e1a83 name:ZooKeeperConnection Watcher:127.0.0.1:55587/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
>   [junit4]   2> 704214 T1723 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
>   [junit4]   2> 704225 T1723 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
>   [junit4]   2> 705028 T1723 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:55609_m_bk
>   [junit4]   2> 705030 T1723 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:55609_m_bk
>   [junit4]   2> 705034 T1769 oascc.ZkStateReader$3.process Updating live nodes... (2)
>   [junit4]   2> 705035 T1756 oascc.ZkStateReader$3.process Updating live nodes... (2)
>   [junit4]   2> 705035 T1747 oascc.ZkStateReader$3.process Updating live nodes... (2)
>   [junit4]   2> 705043 T1770 oasc.ZkController.publish publishing core=collection1 state=down
>   [junit4]   2> 705044 T1770 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
>   [junit4]   2> 705045 T1747 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
>   [junit4]   2> 705046 T1747 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
>   [junit4]   2> 705046 T1747 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
>   [junit4]   2> 705045 T1770 oasc.ZkController.waitForCoreNodeName look for our core node name
>   [junit4]   2> 705048 T1748 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
>   [junit4]   2> 705049 T1748 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
>   [junit4]   2> 	  "operation":"state",
>   [junit4]   2> 	  "state":"down",
>   [junit4]   2> 	  "base_url":"https://127.0.0.1:55609/m_bk",
>   [junit4]   2> 	  "core":"collection1",
>   [junit4]   2> 	  "roles":null,
>   [junit4]   2> 	  "node_name":"127.0.0.1:55609_m_bk",
>   [junit4]   2> 	  "shard":null,
>   [junit4]   2> 	  "collection":"collection1",
>   [junit4]   2> 	  "numShards":"3",
>   [junit4]   2> 	  "core_node_name":null}
>   [junit4]   2> 705049 T1748 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1, shard2, shard3]
>   [junit4]   2> 705049 T1748 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
>   [junit4]   2> 705054 T1747 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
>   [junit4]   2> 705054 T1756 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
>   [junit4]   2> 705054 T1747 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
>   [junit4]   2> 705054 T1769 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
>   [junit4]   2> 705844 T1770 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
>   [junit4]   2> 705844 T1770 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty1-1393439505920\collection1
>   [junit4]   2> 705844 T1770 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
>   [junit4]   2> 705844 T1770 oasc.ZkController.createCollectionZkNode Collection zkNode exists
>   [junit4]   2> 705844 T1770 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
>   [junit4]   2> 705846 T1770 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
>   [junit4]   2> 705847 T1770 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty1-1393439505920\collection1\'
>   [junit4]   2> 705848 T1770 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/C:/Users/JenkinsSlave/workspace/Lucene-Solr-trunk-Windows/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty1-1393439505920/collection1/lib/.svn/' to classloader
>   [junit4]   2> 705850 T1770 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/C:/Users/JenkinsSlave/workspace/Lucene-Solr-trunk-Windows/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty1-1393439505920/collection1/lib/classes/' to classloader
>   [junit4]   2> 705851 T1770 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/C:/Users/JenkinsSlave/workspace/Lucene-Solr-trunk-Windows/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty1-1393439505920/collection1/lib/README' to classloader
>   [junit4]   2> 705898 T1770 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
>   [junit4]   2> 705929 T1770 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
>   [junit4]   2> 705931 T1770 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
>   [junit4]   2> 705940 T1770 oass.IndexSchema.readSchema [collection1] Schema name=test
>   [junit4]   2> 706211 T1770 oass.IndexSchema.readSchema default search field in schema is text
>   [junit4]   2> 706212 T1770 oass.IndexSchema.readSchema unique key field: id
>   [junit4]   2> 706213 T1770 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
>   [junit4]   2> 706219 T1770 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
>   [junit4]   2> 706255 T1770 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
>   [junit4]   2> 706255 T1770 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty1-1393439505920\collection1\, dataDir=.\org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393439503084/jetty1\
>   [junit4]   2> 706255 T1770 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@79d7c4fb
>   [junit4]   2> 706255 T1770 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393439503084\jetty1
>   [junit4]   2> 706257 T1770 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=.\org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393439503084/jetty1\index/
>   [junit4]   2> 706257 T1770 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '.\org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393439503084\jetty1\index' doesn't exist. Creating new index...
>   [junit4]   2> 706257 T1770 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393439503084\jetty1\index
>   [junit4]   2> 706258 T1770 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=39, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
>   [junit4]   2> 706262 T1770 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
>   [junit4]   2> 		commit{dir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393439503084\jetty1\index,segFN=segments_1,generation=1}
>   [junit4]   2> 706262 T1770 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
>   [junit4]   2> 706266 T1770 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
>   [junit4]   2> 706266 T1770 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
>   [junit4]   2> 706266 T1770 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
>   [junit4]   2> 706266 T1770 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
>   [junit4]   2> 706267 T1770 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
>   [junit4]   2> 706267 T1770 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
>   [junit4]   2> 706267 T1770 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
>   [junit4]   2> 706267 T1770 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
>   [junit4]   2> 706267 T1770 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
>   [junit4]   2> 706267 T1770 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
>   [junit4]   2> 706267 T1770 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
>   [junit4]   2> 706267 T1770 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
>   [junit4]   2> 706267 T1770 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
>   [junit4]   2> 706269 T1770 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
>   [junit4]   2> 706269 T1770 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
>   [junit4]   2> 706269 T1770 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
>   [junit4]   2> 706270 T1770 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
>   [junit4]   2> 706274 T1770 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
>   [junit4]   2> 706278 T1770 oasu.CommitTracker.<init> Hard AutoCommit: disabled
>   [junit4]   2> 706278 T1770 oasu.CommitTracker.<init> Soft AutoCommit: disabled
>   [junit4]   2> 706278 T1770 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=1854487036, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
>   [junit4]   2> 706278 T1770 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
>   [junit4]   2> 		commit{dir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393439503084\jetty1\index,segFN=segments_1,generation=1}
>   [junit4]   2> 706278 T1770 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
>   [junit4]   2> 706278 T1770 oass.SolrIndexSearcher.<init> Opening Searcher@5dfc64c0[collection1] main
>   [junit4]   2> 706283 T1771 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5dfc64c0[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
>   [junit4]   2> 706285 T1770 oasc.CoreContainer.registerCore registering core: collection1
>   [junit4]   2> 706285 T1723 oass.SolrDispatchFilter.init user.dir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0
>   [junit4]   2> 706285 T1723 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
>   [junit4]   2> 706285 T1774 oasc.ZkController.register Register replica - core:collection1 address:https://127.0.0.1:55609/m_bk collection:collection1 shard:shard1
>   [junit4]   2> 706288 T1774 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
>   [junit4]   2> 706299 T1774 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
>   [junit4]   2> 706303 T1747 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
>   [junit4]   2> 706304 T1774 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
>   [junit4]   2> 706304 T1774 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
>   [junit4]   2> ASYNC  NEW_CORE C1251 name=collection1 org.apache.solr.core.SolrCore@8a87040 url=https://127.0.0.1:55609/m_bk/collection1 node=127.0.0.1:55609_m_bk C1251_STATE=coll:collection1 core:collection1 props:{state=down, base_url=https://127.0.0.1:55609/m_bk, core=collection1, node_name=127.0.0.1:55609_m_bk}
>   [junit4]   2> 706304 T1774 C1251 P55609 oasc.SyncStrategy.sync Sync replicas to https://127.0.0.1:55609/m_bk/collection1/
>   [junit4]   2> 706304 T1774 C1251 P55609 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
>   [junit4]   2> 706304 T1774 C1251 P55609 oasc.SyncStrategy.syncToMe https://127.0.0.1:55609/m_bk/collection1/ has no replicas
>   [junit4]   2> 706304 T1774 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: https://127.0.0.1:55609/m_bk/collection1/ shard1
>   [junit4]   2> 706304 T1774 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
>   [junit4]   2> 706308 T1748 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
>   [junit4]   2> 706315 T1747 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
>   [junit4]   2> 706318 T1747 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
>   [junit4]   2> 706324 T1747 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
>   [junit4]   2> 706408 T1769 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
>   [junit4]   2> 706408 T1747 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
>   [junit4]   2> 706409 T1756 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
>   [junit4]   2> 706442 T1774 oasc.ZkController.register We are https://127.0.0.1:55609/m_bk/collection1/ and leader is https://127.0.0.1:55609/m_bk/collection1/
>   [junit4]   2> 706442 T1774 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:55609/m_bk
>   [junit4]   2> 706442 T1774 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
>   [junit4]   2> 706442 T1774 oasc.ZkController.publish publishing core=collection1 state=active
>   [junit4]   2> 706442 T1774 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
>   [junit4]   2> 706445 T1747 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
>   [junit4]   2> 706445 T1747 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
>   [junit4]   2> 706445 T1747 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
>   [junit4]   2> 706447 T1774 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
>   [junit4]   2> 706448 T1748 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
>   [junit4]   2> 706448 T1748 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
>   [junit4]   2> 	  "operation":"state",
>   [junit4]   2> 	  "state":"active",
>   [junit4]   2> 	  "base_url":"https://127.0.0.1:55609/m_bk",
>   [junit4]   2> 	  "core":"collection1",
>   [junit4]   2> 	  "roles":null,
>   [junit4]   2> 	  "node_name":"127.0.0.1:55609_m_bk",
>   [junit4]   2> 	  "shard":"shard1",
>   [junit4]   2> 	  "collection":"collection1",
>   [junit4]   2> 	  "numShards":"3",
>   [junit4]   2> 	  "core_node_name":"core_node1"}
>   [junit4]   2> 706453 T1747 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
>   [junit4]   2> 706537 T1769 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
>   [junit4]   2> 706537 T1747 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
>   [junit4]   2> 706537 T1756 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
>   [junit4]   2> 707611 T1723 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 2
>   [junit4]   2> 707611 T1723 oejs.Server.doStart jetty-8.1.10.v20130312
>   [junit4]   2> 707617 T1723 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
>   [junit4]   2> 707619 T1723 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:55618
>   [junit4]   2> 707622 T1723 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
>   [junit4]   2> 707622 T1723 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
>   [junit4]   2> 707622 T1723 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: .\org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty2-1393439509474
>   [junit4]   2> 707622 T1723 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '.\org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty2-1393439509474\'
>   [junit4]   2> 707643 T1723 oasc.ConfigSolr.fromFile Loading container configuration from C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty2-1393439509474\solr.xml
>   [junit4]   2> 707691 T1723 oasc.CoreContainer.<init> New CoreContainer 1544127607
>   [junit4]   2> 707691 T1723 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=.\org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty2-1393439509474\]
>   [junit4]   2> 707692 T1723 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
>   [junit4]   2> 707692 T1723 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
>   [junit4]   2> 707692 T1723 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
>   [junit4]   2> 707692 T1723 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
>   [junit4]   2> 707692 T1723 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
>   [junit4]   2> 707692 T1723 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
>   [junit4]   2> 707692 T1723 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
>   [junit4]   2> 707692 T1723 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
>   [junit4]   2> 707692 T1723 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
>   [junit4]   2> 707699 T1723 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
>   [junit4]   2> 707699 T1723 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
>   [junit4]   2> 707699 T1723 oasc.CoreContainer.load Host Name: 127.0.0.1
>   [junit4]   2> 707699 T1723 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:55587/solr
>   [junit4]   2> 707699 T1723 oasc.ZkController.checkChrootPath zkHost includes chroot
>   [junit4]   2> 707703 T1723 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
>   [junit4]   2> 707715 T1785 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@32562365 name:ZooKeeperConnection Watcher:127.0.0.1:55587 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
>   [junit4]   2> 707716 T1723 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
>   [junit4]   2> 707719 T1723 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
>   [junit4]   2> 707721 T1787 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@261b5054 name:ZooKeeperConnection Watcher:127.0.0.1:55587/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
>   [junit4]   2> 707721 T1723 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
>   [junit4]   2> 707727 T1723 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
>   [junit4]   2> 708530 T1723 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:55618_m_bk
>   [junit4]   2> 708532 T1723 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:55618_m_bk
>   [junit4]   2> 708536 T1769 oascc.ZkStateReader$3.process Updating live nodes... (3)
>   [junit4]   2> 708537 T1747 oascc.ZkStateReader$3.process Updating live nodes... (3)
>   [junit4]   2> 708537 T1756 oascc.ZkStateReader$3.process Updating live nodes... (3)
>   [junit4]   2> 708539 T1787 oascc.ZkStateReader$3.process Updating live nodes... (3)
>   [junit4]   2> 708550 T1788 oasc.ZkController.publish publishing core=collection1 state=down
>   [junit4]   2> 708550 T1788 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
>   [junit4]   2> 708553 T1788 oasc.ZkController.waitForCoreNodeName look for our core node name
>   [junit4]   2> 708553 T1747 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
>   [junit4]   2> 708553 T1747 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
>   [junit4]   2> 708553 T1747 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
>   [junit4]   2> 708555 T1748 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
>   [junit4]   2> 708556 T1748 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
>   [junit4]   2> 	  "operation":"state",
>   [junit4]   2> 	  "state":"down",
>   [junit4]   2> 	  "base_url":"https://127.0.0.1:55618/m_bk",
>   [junit4]   2> 	  "core":"collection1",
>   [junit4]   2> 	  "roles":null,
>   [junit4]   2> 	  "node_name":"127.0.0.1:55618_m_bk",
>   [junit4]   2> 	  "shard":null,
>   [junit4]   2> 	  "collection":"collection1",
>   [junit4]   2> 	  "numShards":"3",
>   [junit4]   2> 	  "core_node_name":null}
>   [junit4]   2> 708556 T1748 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
>   [junit4]   2> 708556 T1748 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard3
>   [junit4]   2> 708561 T1747 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
>   [junit4]   2> 708563 T1787 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
>   [junit4]   2> 708563 T1747 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
>   [junit4]   2> 708563 T1769 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
>   [junit4]   2> 708563 T1756 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
>   [junit4]   2> 709414 T1788 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
>   [junit4]   2> 709414 T1788 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: .\org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty2-1393439509474\collection1
>   [junit4]   2> 709414 T1788 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
>   [junit4]   2> 709414 T1788 oasc.ZkController.createCollectionZkNode Collection zkNode exists
>   [junit4]   2> 709415 T1788 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
>   [junit4]   2> 709417 T1788 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
>   [junit4]   2> 709417 T1788 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '.\org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty2-1393439509474\collection1\'
>   [junit4]   2> 709420 T1788 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/C:/Users/JenkinsSlave/workspace/Lucene-Solr-trunk-Windows/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty2-1393439509474/collection1/lib/.svn/' to classloader
>   [junit4]   2> 709420 T1788 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/C:/Users/JenkinsSlave/workspace/Lucene-Solr-trunk-Windows/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty2-1393439509474/collection1/lib/classes/' to classloader
>   [junit4]   2> 709421 T1788 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/C:/Users/JenkinsSlave/workspace/Lucene-Solr-trunk-Windows/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty2-1393439509474/collection1/lib/README' to classloader
>   [junit4]   2> 709463 T1788 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
>   [junit4]   2> 709499 T1788 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
>   [junit4]   2> 709502 T1788 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
>   [junit4]   2> 709509 T1788 oass.IndexSchema.readSchema [collection1] Schema name=test
>   [junit4]   2> 709811 T1788 oass.IndexSchema.readSchema default search field in schema is text
>   [junit4]   2> 709813 T1788 oass.IndexSchema.readSchema unique key field: id
>   [junit4]   2> 709815 T1788 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
>   [junit4]   2> 709819 T1788 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
>   [junit4]   2> 709858 T1788 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
>   [junit4]   2> 709859 T1788 oasc.SolrCore.<init> [collection1] Opening new SolrCore at .\org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty2-1393439509474\collection1\, dataDir=.\org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393439503084/jetty2\
>   [junit4]   2> 709859 T1788 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@79d7c4fb
>   [junit4]   2> 709860 T1788 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393439503084\jetty2
>   [junit4]   2> 709861 T1788 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=.\org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393439503084/jetty2\index/
>   [junit4]   2> 709862 T1788 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '.\org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393439503084\jetty2\index' doesn't exist. Creating new index...
>   [junit4]   2> 709862 T1788 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393439503084\jetty2\index
>   [junit4]   2> 709863 T1788 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=39, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
>   [junit4]   2> 709868 T1788 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
>   [junit4]   2> 		commit{dir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393439503084\jetty2\index,segFN=segments_1,generation=1}
>   [junit4]   2> 709868 T1788 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
>   [junit4]   2> 709871 T1788 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
>   [junit4]   2> 709871 T1788 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
>   [junit4]   2> 709871 T1788 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
>   [junit4]   2> 709872 T1788 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
>   [junit4]   2> 709872 T1788 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
>   [junit4]   2> 709872 T1788 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
>   [junit4]   2> 709873 T1788 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
>   [junit4]   2> 709873 T1788 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
>   [junit4]   2> 709873 T1788 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
>   [junit4]   2> 709874 T1788 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
>   [junit4]   2> 709874 T1788 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
>   [junit4]   2> 709874 T1788 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
>   [junit4]   2> 709874 T1788 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
>   [junit4]   2> 709875 T1788 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
>   [junit4]   2> 709875 T1788 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
>   [junit4]   2> 709875 T1788 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
>   [junit4]   2> 709875 T1788 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
>   [junit4]   2> 709882 T1788 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
>   [junit4]   2> 709886 T1788 oasu.CommitTracker.<init> Hard AutoCommit: disabled
>   [junit4]   2> 709886 T1788 oasu.CommitTracker.<init> Soft AutoCommit: disabled
>   [junit4]   2> 709887 T1788 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=1854487036, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
>   [junit4]   2> 709887 T1788 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
>   [junit4]   2> 		commit{dir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393439503084\jetty2\index,segFN=segments_1,generation=1}
>   [junit4]   2> 709887 T1788 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
>   [junit4]   2> 709887 T1788 oass.SolrIndexSearcher.<init> Opening Searcher@366b827b[collection1] main
>   [junit4]   2> 709891 T1789 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@366b827b[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
>   [junit4]   2> 709893 T1788 oasc.CoreContainer.registerCore registering core: collection1
>   [junit4]   2> 709894 T1723 oass.SolrDispatchFilter.init user.dir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0
>   [junit4]   2> 709894 T1723 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
>   [junit4]   2> 709895 T1792 oasc.ZkController.register Register replica - core:collection1 address:https://127.0.0.1:55618/m_bk collection:collection1 shard:shard3
>   [junit4]   2> 709898 T1792 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard3/election
>   [junit4]   2> 709907 T1792 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard3
>   [junit4]   2> 709911 T1747 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
>   [junit4]   2> 709913 T1792 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
>   [junit4]   2> 709913 T1792 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
>   [junit4]   2> ASYNC  NEW_CORE C1252 name=collection1 org.apache.solr.core.SolrCore@153326ab url=https://127.0.0.1:55618/m_bk/collection1 node=127.0.0.1:55618_m_bk C1252_STATE=coll:collection1 core:collection1 props:{state=down, base_url=https://127.0.0.1:55618/m_bk, core=collection1, node_name=127.0.0.1:55618_m_bk}
>   [junit4]   2> 709913 T1792 C1252 P55618 oasc.SyncStrategy.sync Sync replicas to https://127.0.0.1:55618/m_bk/collection1/
>   [junit4]   2> 709913 T1792 C1252 P55618 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
>   [junit4]   2> 709913 T1792 C1252 P55618 oasc.SyncStrategy.syncToMe https://127.0.0.1:55618/m_bk/collection1/ has no replicas
>   [junit4]   2> 709913 T1792 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: https://127.0.0.1:55618/m_bk/collection1/ shard3
>   [junit4]   2> 709914 T1792 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard3
>   [junit4]   2> 709917 T1748 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
>   [junit4]   2> 709925 T1747 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
>   [junit4]   2> 709929 T1747 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
>   [junit4]   2> 709936 T1747 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
>   [junit4]   2> 710027 T1787 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
>   [junit4]   2> 710028 T1756 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
>   [junit4]   2> 710028 T1747 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
>   [junit4]   2> 710029 T1769 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
>   [junit4]   2> 710065 T1792 oasc.ZkController.register We are https://127.0.0.1:55618/m_bk/collection1/ and leader is https://127.0.0.1:55618/m_bk/collection1/
>   [junit4]   2> 710065 T1792 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:55618/m_bk
>   [junit4]   2> 710065 T1792 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
>   [junit4]   2> 710065 T1792 oasc.ZkController.publish publishing core=collection1 state=active
>   [junit4]   2> 710065 T1792 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
>   [junit4]   2> 710067 T1747 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
>   [junit4]   2> 710067 T1747 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
>   [junit4]   2> 710067 T1747 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
>   [junit4]   2> 710068 T1792 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
>   [junit4]   2> 710070 T1748 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
>   [junit4]   2> 710071 T1748 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
>   [junit4]   2> 	  "operation":"state",
>   [junit4]   2> 	  "state":"active",
>   [junit4]   2> 	  "base_url":"https://127.0.0.1:55618/m_bk",
>   [junit4]   2> 	  "core":"collection1",
>   [junit4]   2> 	  "roles":null,
>   [junit4]   2> 	  "node_name":"127.0.0.1:55618_m_bk",
>   [junit4]   2> 	  "shard":"shard3",
>   [junit4]   2> 	  "collection":"collection1",
>   [junit4]   2> 	  "numShards":"3",
>   [junit4]   2> 	  "core_node_name":"core_node2"}
>   [junit4]   2> 710076 T1747 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
>   [junit4]   2> 710168 T1769 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
>   [junit4]   2> 710169 T1756 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
>   [junit4]   2> 710169 T1787 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
>   [junit4]   2> 710169 T1747 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
>   [junit4]   2> 711301 T1723 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 3
>   [junit4]   2> 711301 T1723 oejs.Server.doStart jetty-8.1.10.v20130312
>   [junit4]   2> 711307 T1723 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
>   [junit4]   2> 711310 T1723 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:55627
>   [junit4]   2> 711315 T1723 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
>   [junit4]   2> 711315 T1723 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
>   [junit4]   2> 711316 T1723 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty3-1393439513085
>   [junit4]   2> 711316 T1723 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty3-1393439513085\'
>   [junit4]   2> 711341 T1723 oasc.ConfigSolr.fromFile Loading container configuration from C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty3-1393439513085\solr.xml
>   [junit4]   2> 711384 T1723 oasc.CoreContainer.<init> New CoreContainer 1098395650
>   [junit4]   2> 711384 T1723 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty3-1393439513085\]
>   [junit4]   2> 711384 T1723 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
>   [junit4]   2> 711384 T1723 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
>   [junit4]   2> 711384 T1723 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
>   [junit4]   2> 711384 T1723 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
>   [junit4]   2> 711384 T1723 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
>   [junit4]   2> 711384 T1723 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
>   [junit4]   2> 711384 T1723 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
>   [junit4]   2> 711384 T1723 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
>   [junit4]   2> 711384 T1723 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
>   [junit4]   2> 711392 T1723 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
>   [junit4]   2> 711392 T1723 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
>   [junit4]   2> 711392 T1723 oasc.CoreContainer.load Host Name: 127.0.0.1
>   [junit4]   2> 711392 T1723 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:55587/solr
>   [junit4]   2> 711392 T1723 oasc.ZkController.checkChrootPath zkHost includes chroot
>   [junit4]   2> 711396 T1723 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
>   [junit4]   2> 711405 T1803 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@49d1664 name:ZooKeeperConnection Watcher:127.0.0.1:55587 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
>   [junit4]   2> 711405 T1723 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
>   [junit4]   2> 711408 T1725 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x1446f7772f1000a, likely client has closed socket
>   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:220)
>   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
>   [junit4]   2> 	at java.lang.Thread.run(Thread.java:744)
>   [junit4]   2> 
>   [junit4]   2> 711409 T1723 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
>   [junit4]   2> 711412 T1805 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2ff94051 name:ZooKeeperConnection Watcher:127.0.0.1:55587/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
>   [junit4]   2> 711412 T1723 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
>   [junit4]   2> 711419 T1723 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
>   [junit4]   2> 712301 T1723 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:55627_m_bk
>   [junit4]   2> 712303 T1723 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:55627_m_bk
>   [junit4]   2> 712308 T1747 oascc.ZkStateReader$3.process Updating live nodes... (4)
>   [junit4]   2> 712309 T1756 oascc.ZkStateReader$3.process Updating live nodes... (4)
>   [junit4]   2> 712309 T1805 oascc.ZkStateReader$3.process Updating live nodes... (4)
>   [junit4]   2> 712309 T1769 oascc.ZkStateReader$3.process Updating live nodes... (4)
>   [junit4]   2> 712309 T1787 oascc.ZkStateReader$3.process Updating live nodes... (4)
>   [junit4]   2> 712320 T1806 oasc.ZkController.publish publishing core=collection1 state=down
>   [junit4]   2> 712320 T1806 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
>   [junit4]   2> 712322 T1747 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
>   [junit4]   2> 712323 T1747 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
>   [junit4]   2> 712323 T1747 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
>   [junit4]   2> 712322 T1806 oasc.ZkController.waitForCoreNodeName look for our core node name
>   [junit4]   2> 712325 T1748 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
>   [junit4]   2> 712325 T1748 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
>   [junit4]   2> 	  "operation":"state",
>   [junit4]   2> 	  "state":"down",
>   [junit4]   2> 	  "base_url":"https://127.0.0.1:55627/m_bk",
>   [junit4]   2> 	  "core":"collection1",
>   [junit4]   2> 	  "roles":null,
>   [junit4]   2> 	  "node_name":"127.0.0.1:55627_m_bk",
>   [junit4]   2> 	  "shard":null,
>   [junit4]   2> 	  "collection":"collection1",
>   [junit4]   2> 	  "numShards":"3",
>   [junit4]   2> 	  "core_node_name":null}
>   [junit4]   2> 712326 T1748 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
>   [junit4]   2> 712326 T1748 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
>   [junit4]   2> 712330 T1747 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
>   [junit4]   2> 712331 T1787 oas
> 
> [...truncated too long message...]
> 
> insSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393439503084\jetty9 [CachedDir<<refCount=0;path=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393439503084\jetty9;done=false>>]
>   [junit4]   2> 818244 T1922 oasc.CachingDirectoryFactory.close Closing directory: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393439503084\jetty9
>   [junit4]   2> 818244 T1922 oasc.CachingDirectoryFactory.closeCacheValue looking to close C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393439503084\jetty9\index [CachedDir<<refCount=0;path=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393439503084\jetty9\index;done=false>>]
>   [junit4]   2> 818244 T1922 oasc.CachingDirectoryFactory.close Closing directory: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393439503084\jetty9\index
>   [junit4]   2> 818244 T1922 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
>   [junit4]   2> 818244 T1922 oasc.LeaderElector$1.process WARN  org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/election
>   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
>   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
>   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
>   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:257)
>   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:254)
>   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
>   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:254)
>   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:92)
>   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
>   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:137)
>   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
>   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
>   [junit4]   2> 
>   [junit4]   2> 818248 T1962 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
>   [junit4]   2> 818345 T1962 oasc.CachingDirectoryFactory.close Closing StandardDirectoryFactory - 2 directories currently being tracked
>   [junit4]   2> 818345 T1962 oasc.CachingDirectoryFactory.closeCacheValue looking to close C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393439503084\jetty11\index [CachedDir<<refCount=0;path=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393439503084\jetty11\index;done=false>>]
>   [junit4]   2> 818345 T1962 oasc.CachingDirectoryFactory.close Closing directory: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393439503084\jetty11\index
>   [junit4]   2> 818345 T1962 oasc.CachingDirectoryFactory.closeCacheValue looking to close C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393439503084\jetty11 [CachedDir<<refCount=0;path=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393439503084\jetty11;done=false>>]
>   [junit4]   2> 818345 T1962 oasc.CachingDirectoryFactory.close Closing directory: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393439503084\jetty11
>   [junit4]   2> 818345 T1962 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
>   [junit4]   2> 818345 T1962 oasc.LeaderElector$1.process WARN  org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/election
>   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
>   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
>   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
>   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:257)
>   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:254)
>   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
>   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:254)
>   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:92)
>   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
>   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:137)
>   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
>   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
>   [junit4]   2> 
>   [junit4]   2> NOTE: test params are: codec=Lucene41, sim=DefaultSimilarity, locale=ar_IQ, timezone=America/El_Salvador
>   [junit4]   2> NOTE: Windows 7 6.1 amd64/Oracle Corporation 1.7.0_60-ea (64-bit)/cpus=2,threads=1,free=60375552,total=270503936
>   [junit4]   2> NOTE: All tests run in this JVM: [ConnectionManagerTest, PrimitiveFieldTypeTest, TestCollationField, TestNonDefinedSimilarityFactory, TestNoOpRegenerator, DistributedSpellCheckComponentTest, SolrPluginUtilsTest, TestSolrQueryParser, IndexBasedSpellCheckerTest, PathHierarchyTokenizerFactoryTest, HdfsBasicDistributedZk2Test, DocValuesMultiTest, TestFunctionQuery, CSVRequestHandlerTest, BadComponentTest, DistributedDebugComponentTest, DirectUpdateHandlerOptimizeTest, HdfsSyncSliceTest, DirectSolrSpellCheckerTest, TestRandomDVFaceting, URLClassifyProcessorTest, SoftAutoCommitTest, QueryElevationComponentTest, LegacyHTMLStripCharFilterTest, FileBasedSpellCheckerTest, ConvertedLegacyTest, QueryFacetTest, SyncSliceTest, TestQuerySenderListener, DocValuesTest, FieldAnalysisRequestHandlerTest, DisMaxRequestHandlerTest, TestManagedSchema, FileUtilsTest, TestPHPSerializedResponseWriter, HdfsLockFactoryTest, OpenCloseCoreStressTest, TestBinaryResponseWriter, BadIndexSchemaTest, HdfsBasicDistributedZkTest, SuggesterWFSTTest, TestUtils, TestDFRSimilarityFactory, OverseerCollectionProcessorTest, UpdateRequestProcessorFactoryTest, TestAnalyzeInfixSuggestions, TimeZoneUtilsTest, HdfsRecoveryZkTest, PluginInfoTest, RequestHandlersTest, TestFieldTypeResource, TestValueSourceCache, TestCollapseQParserPlugin, SOLR749Test, RecoveryZkTest, TestRandomMergePolicy, PreAnalyzedFieldTest, TestExtendedDismaxParser, JSONWriterTest, TestSuggestSpellingConverter, SimpleFacetsTest, TestOverriddenPrefixQueryForCustomFieldType, HighlighterTest, ScriptEngineTest, ClusterStateTest, ZkControllerTest, BinaryUpdateRequestHandlerTest, TestSolrXmlPersistence, PingRequestHandlerTest, TestRemoteStreaming, DistributedSuggestComponentTest, TestReversedWildcardFilterFactory, TestFoldingMultitermQuery, SortByFunctionTest, TermsComponentTest, DocumentAnalysisRequestHandlerTest, SynonymTokenizerTest, TestDynamicFieldCollectionResource, RegexBoostProcessorTest, TestRandomFaceting, AddSchemaFieldsUpdateProcessorFactoryTest, TestManagedSchemaFieldResource, TestDefaultSimilarityFactory, TestSolrXmlPersistor, SliceStateTest, AbstractAnalyticsStatsTest, EnumFieldTest, IndexSchemaTest, TestInfoStreamLogging, AbstractAnalyticsFacetTest, DistribCursorPagingTest, SolrCoreTest, DistributedQueryElevationComponentTest, EchoParamsTest, ChaosMonkeySafeLeaderTest]
>   [junit4] Completed in 118.97s, 1 test, 1 error <<< FAILURES!
> 
> [...truncated 864 lines...]
> BUILD FAILED
> C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\build.xml:465: The following error occurred while executing this line:
> C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\build.xml:445: The following error occurred while executing this line:
> C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\build.xml:39: The following error occurred while executing this line:
> C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\extra-targets.xml:37: The following error occurred while executing this line:
> C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build.xml:189: The following error occurred while executing this line:
> C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\common-build.xml:490: The following error occurred while executing this line:
> C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\common-build.xml:1274: The following error occurred while executing this line:
> C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\common-build.xml:907: There were test failures: 373 suites, 1606 tests, 1 error, 52 ignored (24 assumptions)
> 
> Total time: 87 minutes 42 seconds
> Build step 'Invoke Ant' marked build as failure
> Description set: Java: 64bit/jdk1.7.0_60-ea-b04 -XX:+UseCompressedOops -XX:+UseSerialGC
> Archiving artifacts
> Recording test results
> Email was triggered for: Failure
> Sending email for trigger: Failure
> 


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


[JENKINS] Lucene-Solr-trunk-Windows (64bit/jdk1.7.0_60-ea-b04) - Build # 3818 - Still Failing!

Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Windows/3818/
Java: 64bit/jdk1.7.0_60-ea-b04 -XX:+UseCompressedOops -XX:+UseSerialGC

1 tests failed.
REGRESSION:  org.apache.solr.cloud.ChaosMonkeySafeLeaderTest.testDistribSearch

Error Message:
KeeperErrorCode = Session expired for /live_nodes

Stack Trace:
org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /live_nodes
	at __randomizedtesting.SeedInfo.seed([DF1F466614BE0A92:5EF9C87E63E16AAE]:0)
	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
	at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:257)
	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:254)
	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
	at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:254)
	at org.apache.solr.common.cloud.ZkStateReader.updateClusterState(ZkStateReader.java:404)
	at org.apache.solr.common.cloud.ZkStateReader.updateClusterState(ZkStateReader.java:230)
	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:144)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.waitForRecoveriesToFinish(AbstractFullDistribZkTestBase.java:730)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.waitForThingsToLevelOut(AbstractFullDistribZkTestBase.java:1602)
	at org.apache.solr.cloud.ChaosMonkeySafeLeaderTest.doTest(ChaosMonkeySafeLeaderTest.java:148)
	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:870)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at java.lang.Thread.run(Thread.java:744)




Build Log:
[...truncated 19360 lines...]
   [junit4] Suite: org.apache.solr.cloud.ChaosMonkeySafeLeaderTest
   [junit4]   2> 699895 T1723 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (true) and clientAuth (false)
   [junit4]   2> 699895 T1723 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /m_bk/
   [junit4]   2> 699898 T1723 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
   [junit4]   2> Creating dataDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\solrtest-ChaosMonkeySafeLeaderTest-1393439503084
   [junit4]   2> 699899 T1723 oasc.ZkTestServer.run STARTING ZK TEST SERVER
   [junit4]   2> 699900 T1724 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
   [junit4]   2> 699973 T1723 oasc.ZkTestServer.run start zk server on port:55587
   [junit4]   2> 699973 T1723 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 699980 T1730 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5cdc87da name:ZooKeeperConnection Watcher:127.0.0.1:55587 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 699980 T1723 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 699980 T1723 oascc.SolrZkClient.makePath makePath: /solr
   [junit4]   2> 699985 T1725 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x1446f7772f10000, likely client has closed socket
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:220)
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:744)
   [junit4]   2> 
   [junit4]   2> 699986 T1723 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 699988 T1732 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3583c568 name:ZooKeeperConnection Watcher:127.0.0.1:55587/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 699988 T1723 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 699988 T1723 oascc.SolrZkClient.makePath makePath: /collections/collection1
   [junit4]   2> 699993 T1723 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
   [junit4]   2> 699996 T1723 oascc.SolrZkClient.makePath makePath: /collections/control_collection
   [junit4]   2> 700000 T1723 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
   [junit4]   2> 700003 T1723 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 700003 T1723 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 700009 T1723 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\schema15.xml to /configs/conf1/schema.xml
   [junit4]   2> 700010 T1723 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
   [junit4]   2> 700013 T1723 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 700013 T1723 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 700017 T1723 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 700017 T1723 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 700021 T1723 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 700021 T1723 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
   [junit4]   2> 700024 T1723 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 700024 T1723 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
   [junit4]   2> 700028 T1723 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 700028 T1723 oascc.SolrZkClient.makePath makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 700031 T1723 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 700031 T1723 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 700035 T1723 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 700036 T1723 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 700039 T1723 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 700039 T1723 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 700041 T1723 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 700043 T1723 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 700048 T1723 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 700051 T1734 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7b314dd8 name:ZooKeeperConnection Watcher:127.0.0.1:55587/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 700051 T1723 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 700054 T1725 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x1446f7772f10002, likely client has closed socket
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:220)
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:744)
   [junit4]   2> 
   [junit4]   2> 701313 T1723 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 701319 T1723 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
   [junit4]   2> 701323 T1723 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:55597
   [junit4]   2> 701326 T1723 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 701326 T1723 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 701326 T1723 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-controljetty-1393439503239
   [junit4]   2> 701326 T1723 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-controljetty-1393439503239\'
   [junit4]   2> 701355 T1723 oasc.ConfigSolr.fromFile Loading container configuration from C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-controljetty-1393439503239\solr.xml
   [junit4]   2> 701393 T1723 oasc.CoreContainer.<init> New CoreContainer 649471709
   [junit4]   2> 701393 T1723 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-controljetty-1393439503239\]
   [junit4]   2> 701395 T1723 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 701395 T1723 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 701395 T1723 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 701395 T1723 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 701395 T1723 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 701395 T1723 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 701395 T1723 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 701395 T1723 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 701395 T1723 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 701403 T1723 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 701403 T1723 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 701403 T1723 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 701403 T1723 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:55587/solr
   [junit4]   2> 701403 T1723 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 701405 T1723 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 701409 T1745 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1734851a name:ZooKeeperConnection Watcher:127.0.0.1:55587 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 701409 T1723 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 701412 T1725 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x1446f7772f10003, likely client has closed socket
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:220)
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:744)
   [junit4]   2> 
   [junit4]   2> 701413 T1723 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 701415 T1747 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5a9ffb81 name:ZooKeeperConnection Watcher:127.0.0.1:55587/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 701415 T1723 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 701418 T1723 oascc.SolrZkClient.makePath makePath: /overseer/queue
   [junit4]   2> 701424 T1723 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
   [junit4]   2> 701431 T1723 oascc.SolrZkClient.makePath makePath: /live_nodes
   [junit4]   2> 701434 T1723 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:55597_m_bk
   [junit4]   2> 701435 T1723 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:55597_m_bk
   [junit4]   2> 701442 T1723 oascc.SolrZkClient.makePath makePath: /overseer_elect
   [junit4]   2> 701446 T1723 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
   [junit4]   2> 701453 T1723 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:55597_m_bk
   [junit4]   2> 701453 T1723 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
   [junit4]   2> 701457 T1723 oasc.Overseer.start Overseer (id=91320451274440708-127.0.0.1:55597_m_bk-n_0000000000) starting
   [junit4]   2> 701463 T1723 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
   [junit4]   2> 701471 T1749 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
   [junit4]   2> 701472 T1723 oascc.SolrZkClient.makePath makePath: /clusterstate.json
   [junit4]   2> 701473 T1749 oasc.OverseerCollectionProcessor.prioritizeOverseerNodes prioritizing overseer nodes
   [junit4]   2> 701477 T1723 oascc.SolrZkClient.makePath makePath: /aliases.json
   [junit4]   2> 701480 T1723 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 701484 T1748 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
   [junit4]   2> 701489 T1750 oasc.ZkController.publish publishing core=collection1 state=down
   [junit4]   2> 701489 T1750 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 701491 T1750 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 701491 T1747 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 701494 T1748 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 701495 T1748 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"https://127.0.0.1:55597/m_bk",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:55597_m_bk",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "collection":"control_collection",
   [junit4]   2> 	  "numShards":"1",
   [junit4]   2> 	  "core_node_name":null}
   [junit4]   2> 701495 T1748 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with shards [shard1]
   [junit4]   2> 701495 T1748 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
   [junit4]   2> 701499 T1747 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 701499 T1747 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 702289 T1750 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
   [junit4]   2> 702289 T1750 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-controljetty-1393439503239\collection1
   [junit4]   2> 702289 T1750 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
   [junit4]   2> 702290 T1750 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 702290 T1750 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
   [junit4]   2> 702292 T1750 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 702292 T1750 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-controljetty-1393439503239\collection1\'
   [junit4]   2> 702296 T1750 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/C:/Users/JenkinsSlave/workspace/Lucene-Solr-trunk-Windows/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-controljetty-1393439503239/collection1/lib/.svn/' to classloader
   [junit4]   2> 702297 T1750 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/C:/Users/JenkinsSlave/workspace/Lucene-Solr-trunk-Windows/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-controljetty-1393439503239/collection1/lib/classes/' to classloader
   [junit4]   2> 702299 T1750 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/C:/Users/JenkinsSlave/workspace/Lucene-Solr-trunk-Windows/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-controljetty-1393439503239/collection1/lib/README' to classloader
   [junit4]   2> 702340 T1750 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
   [junit4]   2> 702372 T1750 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 702374 T1750 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 702381 T1750 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 702645 T1750 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 702646 T1750 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 702647 T1750 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 702652 T1750 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 702683 T1750 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
   [junit4]   2> 702684 T1750 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-controljetty-1393439503239\collection1\, dataDir=.\org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393439503084/control/data\
   [junit4]   2> 702684 T1750 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@79d7c4fb
   [junit4]   2> 702684 T1750 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393439503084\control\data
   [junit4]   2> 702685 T1750 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=.\org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393439503084/control/data\index/
   [junit4]   2> 702686 T1750 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '.\org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393439503084\control\data\index' doesn't exist. Creating new index...
   [junit4]   2> 702686 T1750 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393439503084\control\data\index
   [junit4]   2> 702686 T1750 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=39, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
   [junit4]   2> 702691 T1750 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393439503084\control\data\index,segFN=segments_1,generation=1}
   [junit4]   2> 702691 T1750 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 702694 T1750 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 702694 T1750 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 702695 T1750 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 702695 T1750 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 702695 T1750 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 702695 T1750 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 702695 T1750 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 702695 T1750 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 702695 T1750 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 702696 T1750 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 702696 T1750 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 702696 T1750 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 702696 T1750 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 702697 T1750 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 702697 T1750 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 702697 T1750 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
   [junit4]   2> 702698 T1750 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
   [junit4]   2> 702704 T1750 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 702707 T1750 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 702707 T1750 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 702707 T1750 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=1854487036, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
   [junit4]   2> 702709 T1750 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393439503084\control\data\index,segFN=segments_1,generation=1}
   [junit4]   2> 702709 T1750 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 702709 T1750 oass.SolrIndexSearcher.<init> Opening Searcher@3f73a3dd[collection1] main
   [junit4]   2> 702712 T1751 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3f73a3dd[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
   [junit4]   2> 702713 T1750 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 702714 T1723 oass.SolrDispatchFilter.init user.dir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0
   [junit4]   2> 702714 T1723 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 702714 T1754 oasc.ZkController.register Register replica - core:collection1 address:https://127.0.0.1:55597/m_bk collection:control_collection shard:shard1
   [junit4]   2> 702716 T1754 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 702723 T1723 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 702726 T1756 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@21dc2fac name:ZooKeeperConnection Watcher:127.0.0.1:55587/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 702726 T1723 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 702727 T1754 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
   [junit4]   2> 702730 T1723 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 702731 T1747 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 702732 T1754 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
   [junit4]   2> 702732 T1754 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
   [junit4]   2> ASYNC  NEW_CORE C1250 name=collection1 org.apache.solr.core.SolrCore@430e225d url=https://127.0.0.1:55597/m_bk/collection1 node=127.0.0.1:55597_m_bk C1250_STATE=coll:control_collection core:collection1 props:{state=down, base_url=https://127.0.0.1:55597/m_bk, core=collection1, node_name=127.0.0.1:55597_m_bk}
   [junit4]   2> 702732 T1754 C1250 P55597 oasc.SyncStrategy.sync Sync replicas to https://127.0.0.1:55597/m_bk/collection1/
   [junit4]   2> 702732 T1754 C1250 P55597 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
   [junit4]   2> 702733 T1754 C1250 P55597 oasc.SyncStrategy.syncToMe https://127.0.0.1:55597/m_bk/collection1/ has no replicas
   [junit4]   2> 702733 T1754 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: https://127.0.0.1:55597/m_bk/collection1/ shard1
   [junit4]   2> 702733 T1754 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 702734 T1723 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 702734 T1748 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 702741 T1747 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 702743 T1747 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 702749 T1747 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 702832 T1756 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 702833 T1747 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 702865 T1754 oasc.ZkController.register We are https://127.0.0.1:55597/m_bk/collection1/ and leader is https://127.0.0.1:55597/m_bk/collection1/
   [junit4]   2> 702865 T1754 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:55597/m_bk
   [junit4]   2> 702865 T1754 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
   [junit4]   2> 702865 T1754 oasc.ZkController.publish publishing core=collection1 state=active
   [junit4]   2> 702865 T1754 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 702867 T1747 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 702867 T1747 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 702867 T1747 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 702869 T1754 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 702871 T1748 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 702871 T1748 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"active",
   [junit4]   2> 	  "base_url":"https://127.0.0.1:55597/m_bk",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:55597_m_bk",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "collection":"control_collection",
   [junit4]   2> 	  "numShards":"3",
   [junit4]   2> 	  "core_node_name":"core_node1"}
   [junit4]   2> 702875 T1747 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 702959 T1747 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 702959 T1756 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 704071 T1723 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 1
   [junit4]   2> 704073 T1723 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 704080 T1723 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
   [junit4]   2> 704081 T1723 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:55609
   [junit4]   2> 704084 T1723 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 704084 T1723 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 704084 T1723 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty1-1393439505920
   [junit4]   2> 704084 T1723 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty1-1393439505920\'
   [junit4]   2> 704109 T1723 oasc.ConfigSolr.fromFile Loading container configuration from C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty1-1393439505920\solr.xml
   [junit4]   2> 704152 T1723 oasc.CoreContainer.<init> New CoreContainer 701776337
   [junit4]   2> 704153 T1723 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty1-1393439505920\]
   [junit4]   2> 704154 T1723 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 704154 T1723 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 704154 T1723 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 704154 T1723 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 704155 T1723 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 704155 T1723 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 704155 T1723 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 704155 T1723 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 704155 T1723 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 704162 T1723 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 704162 T1723 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 704163 T1723 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 704163 T1723 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:55587/solr
   [junit4]   2> 704163 T1723 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 704165 T1723 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 704187 T1767 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2c1fd552 name:ZooKeeperConnection Watcher:127.0.0.1:55587 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 704187 T1723 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 704208 T1725 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x1446f7772f10006, likely client has closed socket
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:220)
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:744)
   [junit4]   2> 
   [junit4]   2> 704212 T1723 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 704214 T1769 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@b4e1a83 name:ZooKeeperConnection Watcher:127.0.0.1:55587/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 704214 T1723 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 704225 T1723 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 705028 T1723 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:55609_m_bk
   [junit4]   2> 705030 T1723 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:55609_m_bk
   [junit4]   2> 705034 T1769 oascc.ZkStateReader$3.process Updating live nodes... (2)
   [junit4]   2> 705035 T1756 oascc.ZkStateReader$3.process Updating live nodes... (2)
   [junit4]   2> 705035 T1747 oascc.ZkStateReader$3.process Updating live nodes... (2)
   [junit4]   2> 705043 T1770 oasc.ZkController.publish publishing core=collection1 state=down
   [junit4]   2> 705044 T1770 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 705045 T1747 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 705046 T1747 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 705046 T1747 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 705045 T1770 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 705048 T1748 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 705049 T1748 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"https://127.0.0.1:55609/m_bk",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:55609_m_bk",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"3",
   [junit4]   2> 	  "core_node_name":null}
   [junit4]   2> 705049 T1748 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1, shard2, shard3]
   [junit4]   2> 705049 T1748 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
   [junit4]   2> 705054 T1747 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 705054 T1756 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 705054 T1747 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 705054 T1769 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 705844 T1770 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
   [junit4]   2> 705844 T1770 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty1-1393439505920\collection1
   [junit4]   2> 705844 T1770 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
   [junit4]   2> 705844 T1770 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 705844 T1770 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
   [junit4]   2> 705846 T1770 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 705847 T1770 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty1-1393439505920\collection1\'
   [junit4]   2> 705848 T1770 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/C:/Users/JenkinsSlave/workspace/Lucene-Solr-trunk-Windows/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty1-1393439505920/collection1/lib/.svn/' to classloader
   [junit4]   2> 705850 T1770 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/C:/Users/JenkinsSlave/workspace/Lucene-Solr-trunk-Windows/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty1-1393439505920/collection1/lib/classes/' to classloader
   [junit4]   2> 705851 T1770 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/C:/Users/JenkinsSlave/workspace/Lucene-Solr-trunk-Windows/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty1-1393439505920/collection1/lib/README' to classloader
   [junit4]   2> 705898 T1770 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
   [junit4]   2> 705929 T1770 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 705931 T1770 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 705940 T1770 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 706211 T1770 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 706212 T1770 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 706213 T1770 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 706219 T1770 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 706255 T1770 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
   [junit4]   2> 706255 T1770 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty1-1393439505920\collection1\, dataDir=.\org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393439503084/jetty1\
   [junit4]   2> 706255 T1770 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@79d7c4fb
   [junit4]   2> 706255 T1770 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393439503084\jetty1
   [junit4]   2> 706257 T1770 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=.\org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393439503084/jetty1\index/
   [junit4]   2> 706257 T1770 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '.\org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393439503084\jetty1\index' doesn't exist. Creating new index...
   [junit4]   2> 706257 T1770 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393439503084\jetty1\index
   [junit4]   2> 706258 T1770 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=39, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
   [junit4]   2> 706262 T1770 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393439503084\jetty1\index,segFN=segments_1,generation=1}
   [junit4]   2> 706262 T1770 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 706266 T1770 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 706266 T1770 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 706266 T1770 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 706266 T1770 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 706267 T1770 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 706267 T1770 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 706267 T1770 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 706267 T1770 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 706267 T1770 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 706267 T1770 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 706267 T1770 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 706267 T1770 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 706267 T1770 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 706269 T1770 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 706269 T1770 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 706269 T1770 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
   [junit4]   2> 706270 T1770 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
   [junit4]   2> 706274 T1770 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 706278 T1770 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 706278 T1770 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 706278 T1770 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=1854487036, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
   [junit4]   2> 706278 T1770 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393439503084\jetty1\index,segFN=segments_1,generation=1}
   [junit4]   2> 706278 T1770 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 706278 T1770 oass.SolrIndexSearcher.<init> Opening Searcher@5dfc64c0[collection1] main
   [junit4]   2> 706283 T1771 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5dfc64c0[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
   [junit4]   2> 706285 T1770 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 706285 T1723 oass.SolrDispatchFilter.init user.dir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0
   [junit4]   2> 706285 T1723 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 706285 T1774 oasc.ZkController.register Register replica - core:collection1 address:https://127.0.0.1:55609/m_bk collection:collection1 shard:shard1
   [junit4]   2> 706288 T1774 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
   [junit4]   2> 706299 T1774 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
   [junit4]   2> 706303 T1747 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 706304 T1774 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
   [junit4]   2> 706304 T1774 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
   [junit4]   2> ASYNC  NEW_CORE C1251 name=collection1 org.apache.solr.core.SolrCore@8a87040 url=https://127.0.0.1:55609/m_bk/collection1 node=127.0.0.1:55609_m_bk C1251_STATE=coll:collection1 core:collection1 props:{state=down, base_url=https://127.0.0.1:55609/m_bk, core=collection1, node_name=127.0.0.1:55609_m_bk}
   [junit4]   2> 706304 T1774 C1251 P55609 oasc.SyncStrategy.sync Sync replicas to https://127.0.0.1:55609/m_bk/collection1/
   [junit4]   2> 706304 T1774 C1251 P55609 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
   [junit4]   2> 706304 T1774 C1251 P55609 oasc.SyncStrategy.syncToMe https://127.0.0.1:55609/m_bk/collection1/ has no replicas
   [junit4]   2> 706304 T1774 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: https://127.0.0.1:55609/m_bk/collection1/ shard1
   [junit4]   2> 706304 T1774 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
   [junit4]   2> 706308 T1748 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 706315 T1747 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 706318 T1747 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 706324 T1747 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 706408 T1769 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 706408 T1747 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 706409 T1756 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 706442 T1774 oasc.ZkController.register We are https://127.0.0.1:55609/m_bk/collection1/ and leader is https://127.0.0.1:55609/m_bk/collection1/
   [junit4]   2> 706442 T1774 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:55609/m_bk
   [junit4]   2> 706442 T1774 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
   [junit4]   2> 706442 T1774 oasc.ZkController.publish publishing core=collection1 state=active
   [junit4]   2> 706442 T1774 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 706445 T1747 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 706445 T1747 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 706445 T1747 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 706447 T1774 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 706448 T1748 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 706448 T1748 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"active",
   [junit4]   2> 	  "base_url":"https://127.0.0.1:55609/m_bk",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:55609_m_bk",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"3",
   [junit4]   2> 	  "core_node_name":"core_node1"}
   [junit4]   2> 706453 T1747 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 706537 T1769 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 706537 T1747 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 706537 T1756 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 707611 T1723 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 2
   [junit4]   2> 707611 T1723 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 707617 T1723 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
   [junit4]   2> 707619 T1723 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:55618
   [junit4]   2> 707622 T1723 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 707622 T1723 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 707622 T1723 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: .\org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty2-1393439509474
   [junit4]   2> 707622 T1723 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '.\org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty2-1393439509474\'
   [junit4]   2> 707643 T1723 oasc.ConfigSolr.fromFile Loading container configuration from C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty2-1393439509474\solr.xml
   [junit4]   2> 707691 T1723 oasc.CoreContainer.<init> New CoreContainer 1544127607
   [junit4]   2> 707691 T1723 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=.\org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty2-1393439509474\]
   [junit4]   2> 707692 T1723 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 707692 T1723 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 707692 T1723 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 707692 T1723 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 707692 T1723 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 707692 T1723 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 707692 T1723 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 707692 T1723 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 707692 T1723 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 707699 T1723 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 707699 T1723 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 707699 T1723 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 707699 T1723 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:55587/solr
   [junit4]   2> 707699 T1723 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 707703 T1723 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 707715 T1785 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@32562365 name:ZooKeeperConnection Watcher:127.0.0.1:55587 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 707716 T1723 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 707719 T1723 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 707721 T1787 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@261b5054 name:ZooKeeperConnection Watcher:127.0.0.1:55587/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 707721 T1723 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 707727 T1723 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 708530 T1723 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:55618_m_bk
   [junit4]   2> 708532 T1723 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:55618_m_bk
   [junit4]   2> 708536 T1769 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 708537 T1747 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 708537 T1756 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 708539 T1787 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 708550 T1788 oasc.ZkController.publish publishing core=collection1 state=down
   [junit4]   2> 708550 T1788 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 708553 T1788 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 708553 T1747 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 708553 T1747 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 708553 T1747 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 708555 T1748 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 708556 T1748 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"https://127.0.0.1:55618/m_bk",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:55618_m_bk",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"3",
   [junit4]   2> 	  "core_node_name":null}
   [junit4]   2> 708556 T1748 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
   [junit4]   2> 708556 T1748 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard3
   [junit4]   2> 708561 T1747 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 708563 T1787 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 708563 T1747 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 708563 T1769 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 708563 T1756 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 709414 T1788 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
   [junit4]   2> 709414 T1788 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: .\org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty2-1393439509474\collection1
   [junit4]   2> 709414 T1788 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
   [junit4]   2> 709414 T1788 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 709415 T1788 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
   [junit4]   2> 709417 T1788 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 709417 T1788 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '.\org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty2-1393439509474\collection1\'
   [junit4]   2> 709420 T1788 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/C:/Users/JenkinsSlave/workspace/Lucene-Solr-trunk-Windows/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty2-1393439509474/collection1/lib/.svn/' to classloader
   [junit4]   2> 709420 T1788 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/C:/Users/JenkinsSlave/workspace/Lucene-Solr-trunk-Windows/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty2-1393439509474/collection1/lib/classes/' to classloader
   [junit4]   2> 709421 T1788 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/C:/Users/JenkinsSlave/workspace/Lucene-Solr-trunk-Windows/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty2-1393439509474/collection1/lib/README' to classloader
   [junit4]   2> 709463 T1788 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
   [junit4]   2> 709499 T1788 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 709502 T1788 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 709509 T1788 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 709811 T1788 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 709813 T1788 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 709815 T1788 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 709819 T1788 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 709858 T1788 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
   [junit4]   2> 709859 T1788 oasc.SolrCore.<init> [collection1] Opening new SolrCore at .\org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty2-1393439509474\collection1\, dataDir=.\org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393439503084/jetty2\
   [junit4]   2> 709859 T1788 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@79d7c4fb
   [junit4]   2> 709860 T1788 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393439503084\jetty2
   [junit4]   2> 709861 T1788 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=.\org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393439503084/jetty2\index/
   [junit4]   2> 709862 T1788 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '.\org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393439503084\jetty2\index' doesn't exist. Creating new index...
   [junit4]   2> 709862 T1788 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393439503084\jetty2\index
   [junit4]   2> 709863 T1788 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=39, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
   [junit4]   2> 709868 T1788 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393439503084\jetty2\index,segFN=segments_1,generation=1}
   [junit4]   2> 709868 T1788 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 709871 T1788 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 709871 T1788 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 709871 T1788 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 709872 T1788 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 709872 T1788 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 709872 T1788 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 709873 T1788 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 709873 T1788 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 709873 T1788 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 709874 T1788 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 709874 T1788 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 709874 T1788 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 709874 T1788 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 709875 T1788 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 709875 T1788 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 709875 T1788 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
   [junit4]   2> 709875 T1788 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
   [junit4]   2> 709882 T1788 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 709886 T1788 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 709886 T1788 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 709887 T1788 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=1854487036, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
   [junit4]   2> 709887 T1788 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393439503084\jetty2\index,segFN=segments_1,generation=1}
   [junit4]   2> 709887 T1788 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 709887 T1788 oass.SolrIndexSearcher.<init> Opening Searcher@366b827b[collection1] main
   [junit4]   2> 709891 T1789 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@366b827b[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
   [junit4]   2> 709893 T1788 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 709894 T1723 oass.SolrDispatchFilter.init user.dir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0
   [junit4]   2> 709894 T1723 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 709895 T1792 oasc.ZkController.register Register replica - core:collection1 address:https://127.0.0.1:55618/m_bk collection:collection1 shard:shard3
   [junit4]   2> 709898 T1792 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard3/election
   [junit4]   2> 709907 T1792 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard3
   [junit4]   2> 709911 T1747 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 709913 T1792 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
   [junit4]   2> 709913 T1792 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
   [junit4]   2> ASYNC  NEW_CORE C1252 name=collection1 org.apache.solr.core.SolrCore@153326ab url=https://127.0.0.1:55618/m_bk/collection1 node=127.0.0.1:55618_m_bk C1252_STATE=coll:collection1 core:collection1 props:{state=down, base_url=https://127.0.0.1:55618/m_bk, core=collection1, node_name=127.0.0.1:55618_m_bk}
   [junit4]   2> 709913 T1792 C1252 P55618 oasc.SyncStrategy.sync Sync replicas to https://127.0.0.1:55618/m_bk/collection1/
   [junit4]   2> 709913 T1792 C1252 P55618 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
   [junit4]   2> 709913 T1792 C1252 P55618 oasc.SyncStrategy.syncToMe https://127.0.0.1:55618/m_bk/collection1/ has no replicas
   [junit4]   2> 709913 T1792 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: https://127.0.0.1:55618/m_bk/collection1/ shard3
   [junit4]   2> 709914 T1792 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard3
   [junit4]   2> 709917 T1748 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 709925 T1747 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 709929 T1747 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 709936 T1747 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 710027 T1787 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 710028 T1756 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 710028 T1747 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 710029 T1769 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 710065 T1792 oasc.ZkController.register We are https://127.0.0.1:55618/m_bk/collection1/ and leader is https://127.0.0.1:55618/m_bk/collection1/
   [junit4]   2> 710065 T1792 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:55618/m_bk
   [junit4]   2> 710065 T1792 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
   [junit4]   2> 710065 T1792 oasc.ZkController.publish publishing core=collection1 state=active
   [junit4]   2> 710065 T1792 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 710067 T1747 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 710067 T1747 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 710067 T1747 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 710068 T1792 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 710070 T1748 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 710071 T1748 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"active",
   [junit4]   2> 	  "base_url":"https://127.0.0.1:55618/m_bk",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:55618_m_bk",
   [junit4]   2> 	  "shard":"shard3",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"3",
   [junit4]   2> 	  "core_node_name":"core_node2"}
   [junit4]   2> 710076 T1747 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 710168 T1769 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 710169 T1756 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 710169 T1787 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 710169 T1747 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 711301 T1723 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 3
   [junit4]   2> 711301 T1723 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 711307 T1723 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
   [junit4]   2> 711310 T1723 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:55627
   [junit4]   2> 711315 T1723 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 711315 T1723 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 711316 T1723 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty3-1393439513085
   [junit4]   2> 711316 T1723 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty3-1393439513085\'
   [junit4]   2> 711341 T1723 oasc.ConfigSolr.fromFile Loading container configuration from C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty3-1393439513085\solr.xml
   [junit4]   2> 711384 T1723 oasc.CoreContainer.<init> New CoreContainer 1098395650
   [junit4]   2> 711384 T1723 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty3-1393439513085\]
   [junit4]   2> 711384 T1723 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 711384 T1723 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 711384 T1723 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 711384 T1723 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 711384 T1723 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 711384 T1723 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 711384 T1723 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 711384 T1723 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 711384 T1723 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 711392 T1723 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 711392 T1723 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 711392 T1723 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 711392 T1723 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:55587/solr
   [junit4]   2> 711392 T1723 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 711396 T1723 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 711405 T1803 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@49d1664 name:ZooKeeperConnection Watcher:127.0.0.1:55587 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 711405 T1723 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 711408 T1725 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x1446f7772f1000a, likely client has closed socket
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:220)
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:744)
   [junit4]   2> 
   [junit4]   2> 711409 T1723 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 711412 T1805 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2ff94051 name:ZooKeeperConnection Watcher:127.0.0.1:55587/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 711412 T1723 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 711419 T1723 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 712301 T1723 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:55627_m_bk
   [junit4]   2> 712303 T1723 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:55627_m_bk
   [junit4]   2> 712308 T1747 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 712309 T1756 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 712309 T1805 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 712309 T1769 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 712309 T1787 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 712320 T1806 oasc.ZkController.publish publishing core=collection1 state=down
   [junit4]   2> 712320 T1806 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 712322 T1747 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 712323 T1747 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 712323 T1747 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 712322 T1806 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 712325 T1748 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 712325 T1748 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"https://127.0.0.1:55627/m_bk",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:55627_m_bk",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"3",
   [junit4]   2> 	  "core_node_name":null}
   [junit4]   2> 712326 T1748 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
   [junit4]   2> 712326 T1748 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
   [junit4]   2> 712330 T1747 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 712331 T1787 oas

[...truncated too long message...]

insSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393439503084\jetty9 [CachedDir<<refCount=0;path=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393439503084\jetty9;done=false>>]
   [junit4]   2> 818244 T1922 oasc.CachingDirectoryFactory.close Closing directory: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393439503084\jetty9
   [junit4]   2> 818244 T1922 oasc.CachingDirectoryFactory.closeCacheValue looking to close C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393439503084\jetty9\index [CachedDir<<refCount=0;path=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393439503084\jetty9\index;done=false>>]
   [junit4]   2> 818244 T1922 oasc.CachingDirectoryFactory.close Closing directory: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393439503084\jetty9\index
   [junit4]   2> 818244 T1922 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
   [junit4]   2> 818244 T1922 oasc.LeaderElector$1.process WARN  org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/election
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:257)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:254)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:254)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:92)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:137)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
   [junit4]   2> 
   [junit4]   2> 818248 T1962 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
   [junit4]   2> 818345 T1962 oasc.CachingDirectoryFactory.close Closing StandardDirectoryFactory - 2 directories currently being tracked
   [junit4]   2> 818345 T1962 oasc.CachingDirectoryFactory.closeCacheValue looking to close C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393439503084\jetty11\index [CachedDir<<refCount=0;path=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393439503084\jetty11\index;done=false>>]
   [junit4]   2> 818345 T1962 oasc.CachingDirectoryFactory.close Closing directory: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393439503084\jetty11\index
   [junit4]   2> 818345 T1962 oasc.CachingDirectoryFactory.closeCacheValue looking to close C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393439503084\jetty11 [CachedDir<<refCount=0;path=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393439503084\jetty11;done=false>>]
   [junit4]   2> 818345 T1962 oasc.CachingDirectoryFactory.close Closing directory: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1393439503084\jetty11
   [junit4]   2> 818345 T1962 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
   [junit4]   2> 818345 T1962 oasc.LeaderElector$1.process WARN  org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/election
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:257)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:254)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:254)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:92)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:137)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
   [junit4]   2> 
   [junit4]   2> NOTE: test params are: codec=Lucene41, sim=DefaultSimilarity, locale=ar_IQ, timezone=America/El_Salvador
   [junit4]   2> NOTE: Windows 7 6.1 amd64/Oracle Corporation 1.7.0_60-ea (64-bit)/cpus=2,threads=1,free=60375552,total=270503936
   [junit4]   2> NOTE: All tests run in this JVM: [ConnectionManagerTest, PrimitiveFieldTypeTest, TestCollationField, TestNonDefinedSimilarityFactory, TestNoOpRegenerator, DistributedSpellCheckComponentTest, SolrPluginUtilsTest, TestSolrQueryParser, IndexBasedSpellCheckerTest, PathHierarchyTokenizerFactoryTest, HdfsBasicDistributedZk2Test, DocValuesMultiTest, TestFunctionQuery, CSVRequestHandlerTest, BadComponentTest, DistributedDebugComponentTest, DirectUpdateHandlerOptimizeTest, HdfsSyncSliceTest, DirectSolrSpellCheckerTest, TestRandomDVFaceting, URLClassifyProcessorTest, SoftAutoCommitTest, QueryElevationComponentTest, LegacyHTMLStripCharFilterTest, FileBasedSpellCheckerTest, ConvertedLegacyTest, QueryFacetTest, SyncSliceTest, TestQuerySenderListener, DocValuesTest, FieldAnalysisRequestHandlerTest, DisMaxRequestHandlerTest, TestManagedSchema, FileUtilsTest, TestPHPSerializedResponseWriter, HdfsLockFactoryTest, OpenCloseCoreStressTest, TestBinaryResponseWriter, BadIndexSchemaTest, HdfsBasicDistributedZkTest, SuggesterWFSTTest, TestUtils, TestDFRSimilarityFactory, OverseerCollectionProcessorTest, UpdateRequestProcessorFactoryTest, TestAnalyzeInfixSuggestions, TimeZoneUtilsTest, HdfsRecoveryZkTest, PluginInfoTest, RequestHandlersTest, TestFieldTypeResource, TestValueSourceCache, TestCollapseQParserPlugin, SOLR749Test, RecoveryZkTest, TestRandomMergePolicy, PreAnalyzedFieldTest, TestExtendedDismaxParser, JSONWriterTest, TestSuggestSpellingConverter, SimpleFacetsTest, TestOverriddenPrefixQueryForCustomFieldType, HighlighterTest, ScriptEngineTest, ClusterStateTest, ZkControllerTest, BinaryUpdateRequestHandlerTest, TestSolrXmlPersistence, PingRequestHandlerTest, TestRemoteStreaming, DistributedSuggestComponentTest, TestReversedWildcardFilterFactory, TestFoldingMultitermQuery, SortByFunctionTest, TermsComponentTest, DocumentAnalysisRequestHandlerTest, SynonymTokenizerTest, TestDynamicFieldCollectionResource, RegexBoostProcessorTest, TestRandomFaceting, AddSchemaFieldsUpdateProcessorFactoryTest, TestManagedSchemaFieldResource, TestDefaultSimilarityFactory, TestSolrXmlPersistor, SliceStateTest, AbstractAnalyticsStatsTest, EnumFieldTest, IndexSchemaTest, TestInfoStreamLogging, AbstractAnalyticsFacetTest, DistribCursorPagingTest, SolrCoreTest, DistributedQueryElevationComponentTest, EchoParamsTest, ChaosMonkeySafeLeaderTest]
   [junit4] Completed in 118.97s, 1 test, 1 error <<< FAILURES!

[...truncated 864 lines...]
BUILD FAILED
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\build.xml:465: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\build.xml:445: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\build.xml:39: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\extra-targets.xml:37: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build.xml:189: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\common-build.xml:490: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\common-build.xml:1274: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\common-build.xml:907: There were test failures: 373 suites, 1606 tests, 1 error, 52 ignored (24 assumptions)

Total time: 87 minutes 42 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 64bit/jdk1.7.0_60-ea-b04 -XX:+UseCompressedOops -XX:+UseSerialGC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure