You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Policeman Jenkins Server <je...@thetaphi.de> on 2015/03/30 00:05:13 UTC

[JENKINS] Lucene-Solr-5.x-Linux (64bit/jdk1.7.0_80-ea-b05) - Build # 11979 - Failure!

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-5.x-Linux/11979/
Java: 64bit/jdk1.7.0_80-ea-b05 -XX:+UseCompressedOops -XX:+UseG1GC

2 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.client.solrj.embedded.MergeIndexesEmbeddedTest

Error Message:
Suite timeout exceeded (>= 7200000 msec).

Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
	at __randomizedtesting.SeedInfo.seed([7225193BA072C3CD]:0)


FAILED:  org.apache.solr.client.solrj.embedded.MergeIndexesEmbeddedTest.testMergeMultipleRequest

Error Message:
Test abandoned because suite timeout was reached.

Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
	at __randomizedtesting.SeedInfo.seed([7225193BA072C3CD]:0)




Build Log:
[...truncated 11500 lines...]
   [junit4] Suite: org.apache.solr.client.solrj.embedded.MergeIndexesEmbeddedTest
   [junit4]   2> log4j:WARN No such property [conversionPattern] in org.apache.solr.util.SolrLogLayout.
   [junit4]   2> Creating dataDir: /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MergeIndexesEmbeddedTest 7225193BA072C3CD-001/init-core-data-001
   [junit4]   2> 326 T14 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (false) and clientAuth (false)
   [junit4]   2> 353 T14 oas.SolrTestCaseJ4.setUp ###Starting testMergeMultipleRequest
   [junit4]   2> 376 T14 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/solrj/src/test-files/solrj/solr/multicore/'
   [junit4]   2> 510 T14 oasc.SolrXmlConfig.fromFile Loading container configuration from /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/solrj/src/test-files/solrj/solr/multicore/solr.xml
   [junit4]   2> 588 T14 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/solrj/src/test-files/solrj/solr/multicore
   [junit4]   2> 645 T14 oasc.CoreContainer.<init> New CoreContainer 6844336
   [junit4]   2> 645 T14 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/solrj/src/test-files/solrj/solr/multicore/]
   [junit4]   2> 646 T14 oasc.CoreContainer.load loading shared library: /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/solrj/src/test-files/solrj/solr/multicore/lib
   [junit4]   2> 647 T14 oasc.SolrResourceLoader.addToClassLoader WARN Can't find (or read) directory to add to classloader: lib (resolved as: /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/solrj/src/test-files/solrj/solr/multicore/lib).
   [junit4]   2> 677 T14 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 600000
   [junit4]   2> 678 T14 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: null
   [junit4]   2> 683 T14 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 60000
   [junit4]   2> 684 T14 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 684 T14 oashc.HttpShardHandlerFactory.getParameter Setting maxConnections to: 10000
   [junit4]   2> 685 T14 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 685 T14 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 687 T14 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 687 T14 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 688 T14 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 689 T14 oashc.HttpShardHandlerFactory.getParameter Setting useRetries to: false
   [junit4]   2> 891 T14 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=600000&connTimeout=60000&retry=true
   [junit4]   2> 895 T14 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 897 T14 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 900 T14 oasc.CoreContainer.load Node Name: null
   [junit4]   2> 961 T14 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/solrj/src/test-files/solrj/solr/multicore
   [junit4]   2> 1018 T14 oasc.CoreDescriptor.<init> CORE DESCRIPTOR: {config=solrconfig.xml, loadOnStartup=true, instanceDir=/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/solrj/src/test-files/solrj/solr/multicore/core1, transient=false, name=core1, dataDir=data/, schema=schema.xml, absoluteInstDir=/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/solrj/src/test-files/solrj/solr/multicore/core1/}
   [junit4]   2> 1020 T14 oasc.CorePropertiesLocator.discoverUnder Found core core1 in /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/solrj/src/test-files/solrj/solr/multicore/core1/
   [junit4]   2> 1022 T14 oasc.CoreDescriptor.<init> CORE DESCRIPTOR: {schema=schema.xml, dataDir=data/, loadOnStartup=true, transient=false, config=solrconfig.xml, instanceDir=/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/solrj/src/test-files/solrj/solr/multicore/core0, absoluteInstDir=/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/solrj/src/test-files/solrj/solr/multicore/core0/, name=core0}
   [junit4]   2> 1024 T14 oasc.CorePropertiesLocator.discoverUnder Found core core0 in /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/solrj/src/test-files/solrj/solr/multicore/core0/
   [junit4]   2> 1026 T14 oasc.CorePropertiesLocator.discover Found 2 core definitions
   [junit4]   2> 1029 T16 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/solrj/src/test-files/solrj/solr/multicore/core0/'
   [junit4]   2> 1029 T15 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/solrj/src/test-files/solrj/solr/multicore/core1/'
   [junit4]   2> 1071 T16 oasc.SolrConfig.refreshRequestParams current version of requestparams : -1
   [junit4]   2> 1071 T15 oasc.SolrConfig.refreshRequestParams current version of requestparams : -1
   [junit4]   2> 1111 T15 oasc.SolrConfig.<init> Using Lucene MatchVersion: 5.0.0
   [junit4]   2> 1111 T16 oasc.SolrConfig.<init> Using Lucene MatchVersion: 5.0.0
   [junit4]   2> 1231 T16 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1231 T15 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1242 T16 oass.IndexSchema.readSchema Reading Solr Schema from /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/solrj/src/test-files/solrj/solr/multicore/core0/conf/schema.xml
   [junit4]   2> 1242 T15 oass.IndexSchema.readSchema Reading Solr Schema from /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/solrj/src/test-files/solrj/solr/multicore/core1/conf/schema.xml
   [junit4]   2> 1258 T16 oass.IndexSchema.readSchema [core0] Schema name=example core zero
   [junit4]   2> 1261 T15 oass.IndexSchema.readSchema [core1] Schema name=example core one
   [junit4]   2> 1322 T15 oass.IndexSchema.readSchema default search field in schema is name
   [junit4]   2> 1324 T15 oass.IndexSchema.readSchema query parser default operator is OR
   [junit4]   2> 1326 T15 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 1328 T16 oass.IndexSchema.readSchema default search field in schema is name
   [junit4]   2> 1330 T16 oass.IndexSchema.readSchema query parser default operator is OR
   [junit4]   2> 1331 T15 oasc.CoreContainer.create Creating SolrCore 'core1' using configuration from instancedir /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/solrj/src/test-files/solrj/solr/multicore/core1/
   [junit4]   2> 1332 T16 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 1332 T15 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
   [junit4]   2> 1334 T16 oasc.CoreContainer.create Creating SolrCore 'core0' using configuration from instancedir /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/solrj/src/test-files/solrj/solr/multicore/core0/
   [junit4]   2> 1336 T16 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
   [junit4]   2> 1343 T15 oasc.SolrCore.<init> [core1] Opening new SolrCore at /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/solrj/src/test-files/solrj/solr/multicore/core1/, dataDir=/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MergeIndexesEmbeddedTest 7225193BA072C3CD-001/tempDir-002/
   [junit4]   2> 1343 T16 oasc.SolrCore.<init> [core0] Opening new SolrCore at /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/solrj/src/test-files/solrj/solr/multicore/core0/, dataDir=/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MergeIndexesEmbeddedTest 7225193BA072C3CD-001/tempDir-001/
   [junit4]   2> 1344 T15 oasc.SolrCore.<init> JMX monitoring not detected for core: core1
   [junit4]   2> 1345 T16 oasc.SolrCore.<init> JMX monitoring not detected for core: core0
   [junit4]   2> 1364 T15 oasc.CachingDirectoryFactory.get return new directory for /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MergeIndexesEmbeddedTest 7225193BA072C3CD-001/tempDir-002
   [junit4]   2> 1364 T16 oasc.CachingDirectoryFactory.get return new directory for /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MergeIndexesEmbeddedTest 7225193BA072C3CD-001/tempDir-001
   [junit4]   2> 1366 T15 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MergeIndexesEmbeddedTest 7225193BA072C3CD-001/tempDir-002/index/
   [junit4]   2> 1367 T16 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MergeIndexesEmbeddedTest 7225193BA072C3CD-001/tempDir-001/index/
   [junit4]   2> 1368 T15 oasc.SolrCore.initIndex WARN [core1] Solr index directory '/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MergeIndexesEmbeddedTest 7225193BA072C3CD-001/tempDir-002/index' doesn't exist. Creating new index...
   [junit4]   2> 1369 T16 oasc.SolrCore.initIndex WARN [core0] Solr index directory '/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MergeIndexesEmbeddedTest 7225193BA072C3CD-001/tempDir-001/index' doesn't exist. Creating new index...
   [junit4]   2> 1370 T15 oasc.CachingDirectoryFactory.get return new directory for /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MergeIndexesEmbeddedTest 7225193BA072C3CD-001/tempDir-002/index
   [junit4]   2> 1372 T16 oasc.CachingDirectoryFactory.get return new directory for /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MergeIndexesEmbeddedTest 7225193BA072C3CD-001/tempDir-001/index
   [junit4]   2> 1487 T16 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MergeIndexesEmbeddedTest 7225193BA072C3CD-001/tempDir-001/index,segFN=segments_1,generation=1}
   [junit4]   2> 1490 T16 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 1487 T15 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MergeIndexesEmbeddedTest 7225193BA072C3CD-001/tempDir-002/index,segFN=segments_1,generation=1}
   [junit4]   2> 1492 T15 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 1731 T15 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 1731 T16 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 1786 T15 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 1786 T16 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 1792 T15 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 1792 T16 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 1794 T15 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 1795 T16 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 1796 T15 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 1797 T16 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 1812 T16 oasc.PluginBag.createPlugin requestHandler : '/analysis/field' created with startup=lazy 
   [junit4]   2> 1812 T15 oasc.PluginBag.createPlugin requestHandler : '/analysis/field' created with startup=lazy 
   [junit4]   2> 1813 T16 oasc.RequestHandlers.initHandlersFromConfig Registered paths: /update/json,/schema,,/replication,/update/json/docs,/admin/logging,standard,/analysis/field,/admin/mbeans,/config,/admin/segments,/admin/file,/admin/properties,/update,/admin/ping,/get,/admin/system,/update/csv,/admin/threads,/admin/plugins,/admin/luke
   [junit4]   2> 1813 T15 oasc.RequestHandlers.initHandlersFromConfig Registered paths: /update/json,/schema,,/replication,/update/json/docs,/admin/logging,standard,/analysis/field,/admin/mbeans,/config,/admin/segments,/admin/file,/admin/properties,/update,/admin/ping,/get,/admin/system,/update/csv,/admin/threads,/admin/plugins,/admin/luke
   [junit4]   2> 1814 T16 oasc.RequestHandlers.initHandlersFromConfig WARN no default request handler is registered (either '/select' or 'standard')
   [junit4]   2> 1814 T15 oasc.RequestHandlers.initHandlersFromConfig WARN no default request handler is registered (either '/select' or 'standard')
   [junit4]   2> 1818 T16 oasc.SolrCore.initStatsCache Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 1818 T15 oasc.SolrCore.initStatsCache Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 1832 T15 oasu.UpdateHandler.<init> Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1832 T16 oasu.UpdateHandler.<init> Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1833 T15 oasu.UpdateLog.init Initializing UpdateLog: dataDir=/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MergeIndexesEmbeddedTest 7225193BA072C3CD-001/tempDir-002 defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10
   [junit4]   2> 1834 T16 oasu.UpdateLog.init Initializing UpdateLog: dataDir=/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MergeIndexesEmbeddedTest 7225193BA072C3CD-001/tempDir-001 defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10
   [junit4]   2> 1847 T15 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 1847 T16 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 1848 T15 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 1848 T16 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 1859 T15 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MergeIndexesEmbeddedTest 7225193BA072C3CD-001/tempDir-002/index,segFN=segments_1,generation=1}
   [junit4]   2> 1859 T16 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MergeIndexesEmbeddedTest 7225193BA072C3CD-001/tempDir-001/index,segFN=segments_1,generation=1}
   [junit4]   2> 1860 T15 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 1860 T16 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 1882 T15 oass.SolrIndexSearcher.<init> Opening Searcher@5b9e96a8[core1] main
   [junit4]   2> 1882 T16 oass.SolrIndexSearcher.<init> Opening Searcher@8b51e96[core0] main
   [junit4]   2> 1891 T15 oasr.ManagedResourceStorage.newStorageIO WARN Cannot write to config directory /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/solrj/src/test-files/solrj/solr/multicore/core1/conf; switching to use InMemory storage instead.
   [junit4]   2> 1891 T16 oasr.ManagedResourceStorage.newStorageIO WARN Cannot write to config directory /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/solrj/src/test-files/solrj/solr/multicore/core0/conf; switching to use InMemory storage instead.
   [junit4]   2> 1893 T15 oasr.RestManager.init Initializing RestManager with initArgs: {}
   [junit4]   2> 1893 T16 oasr.RestManager.init Initializing RestManager with initArgs: {}
   [junit4]   2> 1912 T15 oasr.ManagedResourceStorage.load Reading _rest_managed.json using InMemoryStorage
   [junit4]   2> 1912 T15 oasr.RestManager.init Initializing 0 registered ManagedResources
   [junit4]   2> 1920 T16 oasr.ManagedResourceStorage.load Reading _rest_managed.json using InMemoryStorage
   [junit4]   2> 1920 T16 oasr.RestManager.init Initializing 0 registered ManagedResources
   [junit4]   2> 1943 T15 oash.ReplicationHandler.inform Commits will be reserved for  10000
   [junit4]   2> 1945 T17 oasc.SolrCore.registerSearcher [core1] Registered new searcher Searcher@5b9e96a8[core1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1946 T15 oasc.CoreContainer.registerCore registering core: core1
   [junit4]   2> 1947 T16 oash.ReplicationHandler.inform Commits will be reserved for  10000
   [junit4]   2> 1950 T16 oasc.CoreContainer.registerCore registering core: core0
   [junit4]   2> 1950 T18 oasc.SolrCore.registerSearcher [core0] Registered new searcher Searcher@8b51e96[core0] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1954 T14 oascs.MergeIndexesExampleTestBase.setUp CORES=org.apache.solr.core.CoreContainer@686fb0 : [core0, core1]
   [junit4]   2> 1961 T14 oas.SolrTestCaseJ4.tearDown ###Ending testMergeMultipleRequest
   [junit4]   2> 1962 T14 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=6844336
   [junit4]   2> 7199925 T13 ccr.ThreadLeakControl$2.evaluate WARNING Suite execution timed out: org.apache.solr.client.solrj.embedded.MergeIndexesEmbeddedTest
   [junit4]   2> 	==== jstack at approximately timeout time ====
   [junit4]   2> 	"Thread-1" ID=19 WAITING on java.lang.Object@5c64e673
   [junit4]   2> 		at java.lang.Object.wait(Native Method)
   [junit4]   2> 		- waiting on java.lang.Object@5c64e673
   [junit4]   2> 		at java.lang.Object.wait(Object.java:503)
   [junit4]   2> 		at org.apache.solr.core.CloserThread.run(CoreContainer.java:907)
   [junit4]   2> 	
   [junit4]   2> 	"searcherExecutor-7-thread-1" ID=18 WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@6455b768
   [junit4]   2> 		at sun.misc.Unsafe.park(Native Method)
   [junit4]   2> 		- waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@6455b768
   [junit4]   2> 		at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4]   2> 		at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
   [junit4]   2> 		at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]   2> 		at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]   2> 		at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]   2> 		at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]   2> 		at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 	
   [junit4]   2> 	"searcherExecutor-6-thread-1" ID=17 WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@377c404e
   [junit4]   2> 		at sun.misc.Unsafe.park(Native Method)
   [junit4]   2> 		- waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@377c404e
   [junit4]   2> 		at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4]   2> 		at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
   [junit4]   2> 		at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]   2> 		at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]   2> 		at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]   2> 		at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]   2> 		at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 	
   [junit4]   2> 	"TEST-MergeIndexesEmbeddedTest.testMergeMultipleRequest-seed#[7225193BA072C3CD]" ID=14 WAITING on org.apache.solr.core.CloserThread@3e61dd57
   [junit4]   2> 		at java.lang.Object.wait(Native Method)
   [junit4]   2> 		- waiting on org.apache.solr.core.CloserThread@3e61dd57
   [junit4]   2> 		at java.lang.Thread.join(Thread.java:1281)
   [junit4]   2> 		at java.lang.Thread.join(Thread.java:1355)
   [junit4]   2> 		at org.apache.solr.core.CoreContainer.shutdown(CoreContainer.java:365)
   [junit4]   2> 		at org.apache.solr.client.solrj.MergeIndexesExampleTestBase.tearDown(MergeIndexesExampleTestBase.java:87)
   [junit4]   2> 		at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   2> 		at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   [junit4]   2> 		at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2> 		at java.lang.reflect.Method.invoke(Method.java:606)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:894)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
   [junit4]   2> 		at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
   [junit4]   2> 		at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2> 		at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
   [junit4]   2> 		at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2> 		at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
   [junit4]   2> 		at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 		at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 		at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   2> 		at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2> 		at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2> 		at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   2> 		at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 	
   [junit4]   2> 	"SUITE-MergeIndexesEmbeddedTest-seed#[7225193BA072C3CD]" ID=13 RUNNABLE
   [junit4]   2> 		at sun.management.ThreadImpl.dumpThreads0(Native Method)
   [junit4]   2> 		at sun.management.ThreadImpl.dumpAllThreads(ThreadImpl.java:446)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.ThreadLeakControl.formatThreadStacksFull(ThreadLeakControl.java:671)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.ThreadLeakControl.access$900(ThreadLeakControl.java:62)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.ThreadLeakControl$2.evaluate(ThreadLeakControl.java:412)
   [junit4]   2> 		- locked java.lang.Object@b0a9ac0
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:651)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.RandomizedRunner.access$200(RandomizedRunner.java:138)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.RandomizedRunner$1.run(RandomizedRunner.java:568)
   [junit4]   2> 	
   [junit4]   2> 	"JUnit4-serializer-daemon" ID=10 TIMED_WAITING
   [junit4]   2> 		at java.lang.Thread.sleep(Native Method)
   [junit4]   2> 		at com.carrotsearch.ant.tasks.junit4.events.Serializer$1.run(Serializer.java:47)
   [junit4]   2> 	
   [junit4]   2> 	"Signal Dispatcher" ID=5 RUNNABLE
   [junit4]   2> 	
   [junit4]   2> 	"Finalizer" ID=3 WAITING on java.lang.ref.ReferenceQueue$Lock@6535b8e9
   [junit4]   2> 		at java.lang.Object.wait(Native Method)
   [junit4]   2> 		- waiting on java.lang.ref.ReferenceQueue$Lock@6535b8e9
   [junit4]   2> 		at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:135)
   [junit4]   2> 		at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:151)
   [junit4]   2> 		at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:209)
   [junit4]   2> 	
   [junit4]   2> 	"Reference Handler" ID=2 WAITING on java.lang.ref.Reference$Lock@29fefce8
   [junit4]   2> 		at java.lang.Object.wait(Native Method)
   [junit4]   2> 		- waiting on java.lang.ref.Reference$Lock@29fefce8
   [junit4]   2> 		at java.lang.Object.wait(Object.java:503)
   [junit4]   2> 		at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:133)
   [junit4]   2> 	
   [junit4]   2> 	"main" ID=1 WAITING on com.carrotsearch.randomizedtesting.RandomizedRunner$1@3afe1ac8
   [junit4]   2> 		at java.lang.Object.wait(Native Method)
   [junit4]   2> 		- waiting on com.carrotsearch.randomizedtesting.RandomizedRunner$1@3afe1ac8
   [junit4]   2> 		at java.lang.Thread.join(Thread.java:1281)
   [junit4]   2> 		at java.lang.Thread.join(Thread.java:1355)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:578)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.RandomizedRunner.run(RandomizedRunner.java:444)
   [junit4]   2> 		at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.execute(SlaveMain.java:199)
   [junit4]   2> 		at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.main(SlaveMain.java:310)
   [junit4]   2> 		at com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe.main(SlaveMainSafe.java:12)
   [junit4]   2> 	
   [junit4]   2> 	^^==============================================
   [junit4]   2> 	
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=MergeIndexesEmbeddedTest -Dtests.method=testMergeMultipleRequest -Dtests.seed=7225193BA072C3CD -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=sk -Dtests.timezone=SST -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR   7200s J1 | MergeIndexesEmbeddedTest.testMergeMultipleRequest <<<
   [junit4]    > Throwable #1: java.lang.Exception: Test abandoned because suite timeout was reached.
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([7225193BA072C3CD]:0)
   [junit4]   2> 7199948 T13 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 4 leaked thread(s).
   [junit4]   2> 7220012 T13 ccr.ThreadLeakControl.checkThreadLeaks SEVERE 4 threads leaked from SUITE scope at org.apache.solr.client.solrj.embedded.MergeIndexesEmbeddedTest: 
   [junit4]   2> 	   1) Thread[id=18, name=searcherExecutor-7-thread-1, state=WAITING, group=TGRP-MergeIndexesEmbeddedTest]
   [junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
   [junit4]   2> 	        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
   [junit4]   2> 	        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]   2> 	        at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 	   2) Thread[id=19, name=Thread-1, state=WAITING, group=TGRP-MergeIndexesEmbeddedTest]
   [junit4]   2> 	        at java.lang.Object.wait(Native Method)
   [junit4]   2> 	        at java.lang.Object.wait(Object.java:503)
   [junit4]   2> 	        at org.apache.solr.core.CloserThread.run(CoreContainer.java:907)
   [junit4]   2> 	   3) Thread[id=14, name=TEST-MergeIndexesEmbeddedTest.testMergeMultipleRequest-seed#[7225193BA072C3CD], state=WAITING, group=TGRP-MergeIndexesEmbeddedTest]
   [junit4]   2> 	        at java.lang.Object.wait(Native Method)
   [junit4]   2> 	        at java.lang.Thread.join(Thread.java:1281)
   [junit4]   2> 	        at java.lang.Thread.join(Thread.java:1355)
   [junit4]   2> 	        at org.apache.solr.core.CoreContainer.shutdown(CoreContainer.java:365)
   [junit4]   2> 	        at org.apache.solr.client.solrj.MergeIndexesExampleTestBase.tearDown(MergeIndexesExampleTestBase.java:87)
   [junit4]   2> 	        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   2> 	        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   [junit4]   2> 	        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2> 	        at java.lang.reflect.Method.invoke(Method.java:606)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:894)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
   [junit4]   2> 	        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
   [junit4]   2> 	        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2> 	        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
   [junit4]   2> 	        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2> 	        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
   [junit4]   2> 	        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   2> 	        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2> 	        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2> 	        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   2> 	        at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 	   4) Thread[id=17, name=searcherExecutor-6-thread-1, state=WAITING, group=TGRP-MergeIndexesEmbeddedTest]
   [junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
   [junit4]   2> 	        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
   [junit4]   2> 	        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]   2> 	        at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 7220014 T13 ccr.ThreadLeakControl.tryToInterruptAll Starting to interrupt leaked threads:
   [junit4]   2> 	   1) Thread[id=18, name=searcherExecutor-7-thread-1, state=WAITING, group=TGRP-MergeIndexesEmbeddedTest]
   [junit4]   2> 	   2) Thread[id=19, name=Thread-1, state=WAITING, group=TGRP-MergeIndexesEmbeddedTest]
   [junit4]   2> 	   3) Thread[id=14, name=TEST-MergeIndexesEmbeddedTest.testMergeMultipleRequest-seed#[7225193BA072C3CD], state=WAITING, group=TGRP-MergeIndexesEmbeddedTest]
   [junit4]   2> 	   4) Thread[id=17, name=searcherExecutor-6-thread-1, state=WAITING, group=TGRP-MergeIndexesEmbeddedTest]
   [junit4]   2> 7220190 T14 oasc.SolrCore.close [core1]  CLOSING SolrCore org.apache.solr.core.SolrCore@29f29436
   [junit4]   2> 7220191 T14 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0,transaction_logs_total_size=0,transaction_logs_total_number=0}
   [junit4]   2> 7220192 T14 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
   [junit4]   2> 7220192 T14 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
   [junit4]   2> 7220193 T14 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
   [junit4]   2> 7220199 T14 oasc.SolrCore.closeSearcher [core1] Closing main searcher on request.
   [junit4]   2> 7220200 T14 oasc.CachingDirectoryFactory.close Closing StandardDirectoryFactory - 2 directories currently being tracked
   [junit4]   2> 7220201 T14 oasc.CachingDirectoryFactory.closeCacheValue looking to close /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MergeIndexesEmbeddedTest 7225193BA072C3CD-001/tempDir-002/index [CachedDir<<refCount=0;path=/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MergeIndexesEmbeddedTest 7225193BA072C3CD-001/tempDir-002/index;done=false>>]
   [junit4]   2> 7220202 T14 oasc.CachingDirectoryFactory.close Closing directory: /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MergeIndexesEmbeddedTest 7225193BA072C3CD-001/tempDir-002/index
   [junit4]   2> 7220203 T14 oasc.CachingDirectoryFactory.closeCacheValue looking to close /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MergeIndexesEmbeddedTest 7225193BA072C3CD-001/tempDir-002 [CachedDir<<refCount=0;path=/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MergeIndexesEmbeddedTest 7225193BA072C3CD-001/tempDir-002;done=false>>]
   [junit4]   2> 7220204 T14 oasc.CachingDirectoryFactory.close Closing directory: /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MergeIndexesEmbeddedTest 7225193BA072C3CD-001/tempDir-002
   [junit4]   2> 7220204 T14 oasc.SolrCore.close [core0]  CLOSING SolrCore org.apache.solr.core.SolrCore@aad9e9d
   [junit4]   2> 7220205 T14 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0,transaction_logs_total_size=0,transaction_logs_total_number=0}
   [junit4]   2> 7220206 T14 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
   [junit4]   2> 7220206 T14 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
   [junit4]   2> 7220207 T14 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
   [junit4]   2> 7220209 T14 oasc.SolrCore.closeSearcher [core0] Closing main searcher on request.
   [junit4]   2> 7220210 T14 oasc.CachingDirectoryFactory.close Closing StandardDirectoryFactory - 2 directories currently being tracked
   [junit4]   2> 7220211 T14 oasc.CachingDirectoryFactory.closeCacheValue looking to close /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MergeIndexesEmbeddedTest 7225193BA072C3CD-001/tempDir-001/index [CachedDir<<refCount=0;path=/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MergeIndexesEmbeddedTest 7225193BA072C3CD-001/tempDir-001/index;done=false>>]
   [junit4]   2> 7220211 T14 oasc.CachingDirectoryFactory.close Closing directory: /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MergeIndexesEmbeddedTest 7225193BA072C3CD-001/tempDir-001/index
   [junit4]   2> 7220212 T14 oasc.CachingDirectoryFactory.closeCacheValue looking to close /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MergeIndexesEmbeddedTest 7225193BA072C3CD-001/tempDir-001 [CachedDir<<refCount=0;path=/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MergeIndexesEmbeddedTest 7225193BA072C3CD-001/tempDir-001;done=false>>]
   [junit4]   2> 7220213 T14 oasc.CachingDirectoryFactory.close Closing directory: /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MergeIndexesEmbeddedTest 7225193BA072C3CD-001/tempDir-001
   [junit4]   2> 7220223 T14 oas.SolrTestCaseJ4.setUp ###Starting testMergeIndexesByCoreName
   [junit4]   2> 7220226 T14 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/solrj/src/test-files/solrj/solr/multicore/'
   [junit4]   2> 7220255 T14 oasc.SolrXmlConfig.fromFile Loading container configuration from /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/solrj/src/test-files/solrj/solr/multicore/solr.xml
   [junit4]   2> 7220275 T14 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/solrj/src/test-files/solrj/solr/multicore
   [junit4]   2> 7220276 T14 oasc.CoreContainer.<init> New CoreContainer 1859197274
   [junit4]   2> 7220276 T14 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/solrj/src/test-files/solrj/solr/multicore/]
   [junit4]   2> 7220277 T14 oasc.CoreContainer.load loading shared library: /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/solrj/src/test-files/solrj/solr/multicore/lib
   [junit4]   2> 7220278 T14 oasc.SolrResourceLoader.addToClassLoader WARN Can't find (or read) directory to add to classloader: lib (resolved as: /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/solrj/src/test-files/solrj/solr/multicore/lib).
   [junit4]   2> 7220289 T14 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 600000
   [junit4]   2> 7220290 T14 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: null
   [junit4]   2> 7220290 T14 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 60000
   [junit4]   2> 7220291 T14 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 7220291 T14 oashc.HttpShardHandlerFactory.getParameter Setting maxConnections to: 10000
   [junit4]   2> 7220292 T14 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 7220292 T14 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 7220293 T14 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 7220293 T14 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 7220294 T14 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 7220294 T14 oashc.HttpShardHandlerFactory.getParameter Setting useRetries to: false
   [junit4]   2> 7220295 T14 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=600000&connTimeout=60000&retry=true
   [junit4]   2> 7220296 T14 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 7220296 T14 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 7220297 T14 oasc.CoreContainer.load Node Name: null
   [junit4]   2> 7220298 T14 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/solrj/src/test-files/solrj/solr/multicore
   [junit4]   2> 7220300 T14 oasc.CoreDescriptor.<init> CORE DESCRIPTOR: {absoluteInstDir=/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/solrj/src/test-files/solrj/solr/multicore/core1/, transient=false, instanceDir=/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/solrj/src/test-files/solrj/solr/multicore/core1, dataDir=data/, config=solrconfig.xml, name=core1, loadOnStartup=true, schema=schema.xml}
   [junit4]   2> 7220300 T14 oasc.CorePropertiesLocator.discoverUnder Found core core1 in /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/solrj/src/test-files/solrj/solr/multicore/core1/
   [junit4]   2> 7220301 T14 oasc.CoreDescriptor.<init> CORE DESCRIPTOR: {name=core0, config=solrconfig.xml, dataDir=data/, transient=false, instanceDir=/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/solrj/src/test-files/solrj/solr/multicore/core0, absoluteInstDir=/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/solrj/src/test-files/solrj/solr/multicore/core0/, schema=schema.xml, loadOnStartup=true}
   [junit4]   2> 7220302 T14 oasc.CorePropertiesLocator.discoverUnder Found core core0 in /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/solrj/src/test-files/solrj/solr/multicore/core0/
   [junit4]   2> 7220303 T14 oasc.CorePropertiesLocator.discover Found 2 core definitions
   [junit4]   2> 7220304 T20 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/solrj/src/test-files/solrj/solr/multicore/core1/'
   [junit4]   2> 7220304 T21 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/solrj/src/test-files/solrj/solr/multicore/core0/'
   [junit4]   2> 7220331 T20 oasc.SolrConfig.refreshRequestParams current version of requestparams : -1
   [junit4]   2> 7220335 T21 oasc.SolrConfig.refreshRequestParams current version of requestparams : -1
   [junit4]   2> 7220352 T20 oasc.SolrConfig.<init> Using Lucene MatchVersion: 5.0.0
   [junit4]   2> 7220356 T21 oasc.SolrConfig.<init> Using Lucene MatchVersion: 5.0.0
   [junit4]   2> 7220426 T20 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 7220427 T20 oass.IndexSchema.readSchema Reading Solr Schema from /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/solrj/src/test-files/solrj/solr/multicore/core1/conf/schema.xml
   [junit4]   2> 7220435 T20 oass.IndexSchema.readSchema [core1] Schema name=example core one
   [junit4]   2> 7220442 T21 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 7220443 T21 oass.IndexSchema.readSchema Reading Solr Schema from /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/solrj/src/test-files/solrj/solr/multicore/core0/conf/schema.xml
   [junit4]   2> 7220447 T20 oass.IndexSchema.readSchema default search field in schema is name
   [junit4]   2> 7220448 T20 oass.IndexSchema.readSchema query parser default operator is OR
   [junit4]   2> 7220449 T20 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 7220450 T20 oasc.CoreContainer.create Creating SolrCore 'core1' using configuration from instancedir /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/solrj/src/test-files/solrj/solr/multicore/core1/
   [junit4]   2> 7220450 T20 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
   [junit4]   2> 7220451 T20 oasc.SolrCore.<init> [core1] Opening new SolrCore at /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/solrj/src/test-files/solrj/solr/multicore/core1/, dataDir=/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MergeIndexesEmbeddedTest 7225193BA072C3CD-001/tempDir-004/
   [junit4]   2> 7220451 T20 oasc.SolrCore.<init> JMX monitoring not detected for core: core1
   [junit4]   2> 7220452 T20 oasc.CachingDirectoryFactory.get return new directory for /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MergeIndexesEmbeddedTest 7225193BA072C3CD-001/tempDir-004
   [junit4]   2> 7220452 T20 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MergeIndexesEmbeddedTest 7225193BA072C3CD-001/tempDir-004/index/
   [junit4]   2> 7220453 T20 oasc.SolrCore.initIndex WARN [core1] Solr index directory '/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MergeIndexesEmbeddedTest 7225193BA072C3CD-001/tempDir-004/index' doesn't exist. Creating new index...
   [junit4]   2> 7220453 T20 oasc.CachingDirectoryFactory.get return new directory for /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MergeIndexesEmbeddedTest 7225193BA072C3CD-001/tempDir-004/index
   [junit4]   2> 7220454 T21 oass.IndexSchema.readSchema [core0] Schema name=example core zero
   [junit4]   2> 7220457 T20 oasu.SolrIndexWriter.close ERROR Error closing IndexWriter java.nio.channels.ClosedByInterruptException
   [junit4]   2> 	at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:202)
   [junit4]   2> 	at sun.nio.ch.FileChannelImpl.write(FileChannelImpl.java:210)
   [junit4]   2> 	at java.nio.channels.Channels.writeFullyImpl(Channels.java:78)
   [junit4]   2> 	at java.nio.channels.Channels.writeFully(Channels.java:101)
   [junit4]   2> 	at java.nio.channels.Channels.access$000(Channels.java:61)
   [junit4]   2> 	at java.nio.channels.Channels$1.write(Channels.java:174)
   [junit4]   2> 	at org.apache.lucene.store.FSDirectory$FSIndexOutput$1.write(FSDirectory.java:268)
   [junit4]   2> 	at java.util.zip.CheckedOutputStream.write(CheckedOutputStream.java:73)
   [junit4]   2> 	at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
   [junit4]   2> 	at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
   [junit4]   2> 	at org.apache.lucene.store.OutputStreamIndexOutput.getChecksum(OutputStreamIndexOutput.java:80)
   [junit4]   2> 	at org.apache.lucene.codecs.CodecUtil.writeFooter(CodecUtil.java:310)
   [junit4]   2> 	at org.apache.lucene.index.SegmentInfos.write(SegmentInfos.java:507)
   [junit4]   2> 	at org.apache.lucene.index.SegmentInfos.prepareCommit(SegmentInfos.java:731)
   [junit4]   2> 	at org.apache.lucene.index.IndexWriter.startCommit(IndexWriter.java:4290)
   [junit4]   2> 	at org.apache.lucene.index.IndexWriter.prepareCommitInternal(IndexWriter.java:2740)
   [junit4]   2> 	at org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:2843)
   [junit4]   2> 	at org.apache.lucene.index.IndexWriter.shutdown(IndexWriter.java:968)
   [junit4]   2> 	at org.apache.lucene.index.IndexWriter.close(IndexWriter.java:1011)
   [junit4]   2> 	at org.apache.solr.update.SolrIndexWriter.close(SolrIndexWriter.java:130)
   [junit4]   2> 	at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:539)
   [junit4]   2> 	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:796)
   [junit4]   2> 	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:652)
   [junit4]   2> 	at org.apache.solr.core.CoreContainer.create(CoreContainer.java:518)
   [junit4]   2> 	at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:283)
   [junit4]   2> 	at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:277)
   [junit4]   2> 	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 
   [junit4]   2> 7220460 T20 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 7220463 T20 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 7220464 T20 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 7220466 T20 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 7220467 T20 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 7220469 T21 oass.IndexSchema.readSchema default search field in schema is name
   [junit4]   2> 7220470 T20 oasc.PluginBag.createPlugin requestHandler : '/analysis/field' created with startup=lazy 
   [junit4]   2> 7220470 T20 oasc.RequestHandlers.initHandlersFromConfig Registered paths: /update/json,/schema,,/replication,/update/json/docs,/admin/logging,standard,/analysis/field,/admin/mbeans,/config,/admin/segments,/admin/file,/admin/properties,/update,/admin/ping,/get,/admin/system,/update/csv,/admin/threads,/admin/plugins,/admin/luke
   [junit4]   2> 7220470 T21 oass.IndexSchema.readSchema query parser default operator is OR
   [junit4]   2> 7220470 T20 oasc.RequestHandlers.initHandlersFromConfig WARN no default request handler is registered (either '/select' or 'standard')
   [junit4]   2> 7220472 T21 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 7220472 T20 oasc.SolrCore.initStatsCache Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 7220473 T20 oasu.UpdateHandler.<init> Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 7220473 T21 oasc.CoreContainer.create Creating SolrCore 'core0' using configuration from instancedir /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/solrj/src/test-files/solrj/solr/multicore/core0/
   [junit4]   2> 7220473 T20 oasu.UpdateLog.init Initializing UpdateLog: dataDir=/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MergeIndexesEmbeddedTest 7225193BA072C3CD-001/tempDir-004 defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10
   [junit4]   2> 7220474 T21 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
   [junit4]   2> 7220475 T20 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 7220475 T21 oasc.SolrCore.<init> [core0] Opening new SolrCore at /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/solrj/src/test-files/solrj/solr/multicore/core0/, dataDir=/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MergeIndexesEmbeddedTest 7225193BA072C3CD-001/tempDir-003/
   [junit4]   2> 7220476 T21 oasc.SolrCore.<init> JMX monitoring not detected for core: core0
   [junit4]   2> 7220475 T20 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 7220477 T21 oasc.CachingDirectoryFactory.get return new directory for /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MergeIndexesEmbeddedTest 7225193BA072C3CD-001/tempDir-003
   [junit4]   2> 7220478 T21 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MergeIndexesEmbeddedTest 7225193BA072C3CD-001/tempDir-003/index/
   [junit4]   2> 7220478 T20 oasc.CachingDirectoryFactory.closeCacheValue looking to close /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MergeIndexesEmbeddedTest 7225193BA072C3CD-001/tempDir-004/index [CachedDir<<refCount=0;path=/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MergeIndexesEmbeddedTest 7225193BA072C3CD-001/tempDir-004/index;done=true>>]
   [junit4]   2> 7220478 T21 oasc.SolrCore.initIndex WARN [core0] Solr index directory '/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MergeIndexesEmbeddedTest 7225193BA072C3CD-001/tempDir-003/index' doesn't exist. Creating new index...
   [junit4]   2> 7220478 T20 oasc.CachingDirectoryFactory.close Closing directory: /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MergeIndexesEmbeddedTest 7225193BA072C3CD-001/tempDir-004/index
   [junit4]   2> 7220479 T21 oasc.CachingDirectoryFactory.get return new directory for /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MergeIndexesEmbeddedTest 7225193BA072C3CD-001/tempDir-003/index
   [junit4]   2> 7220480 T20 oasc.SolrCore.close [core1]  CLOSING SolrCore org.apache.solr.core.SolrCore@4d1bc5d7
   [junit4]   2> 7220481 T20 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0,transaction_logs_total_size=0,transaction_logs_total_number=0}
   [junit4]   2> 7220481 T20 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
   [junit4]   2> 7220481 T20 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
   [junit4]   2> 7220482 T20 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
   [junit4]   2> 7220482 T20 oasc.SolrCore.closeSearcher [core1] Closing main searcher on request.
   [junit4]   2> 7220482 T20 oasc.CachingDirectoryFactory.close Closing StandardDirectoryFactory - 1 directories currently being tracked
   [junit4]   2> 7220483 T20 oasc.CachingDirectoryFactory.closeCacheValue looking to close /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MergeIndexesEmbeddedTest 7225193BA072C3CD-001/tempDir-004 [CachedDir<<refCount=0;path=/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MergeIndexesEmbeddedTest 7225193BA072C3CD-001/tempDir-004;done=false>>]
   [junit4]   2> 7220483 T20 oasc.CachingDirectoryFactory.close Closing directory: /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MergeIndexesEmbeddedTest 7225193BA072C3CD-001/tempDir-004
   [junit4]   2> 7220484 T20 oasc.CoreContainer.create ERROR Error creating core [core1]: Error opening new searcher org.apache.solr.common.SolrException: Error opening new searcher
   [junit4]   2> 	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:885)
   [junit4]   2> 	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:652)
   [junit4]   2> 	at org.apache.solr.core.CoreContainer.create(CoreContainer.java:518)
   [junit4]   2> 	at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:283)
   [junit4]   2> 	at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:277)
   [junit4]   2> 	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> Caused by: org.apache.solr.common.SolrException: Error opening new searcher
   [junit4]   2> 	at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1577)
   [junit4]   2> 	at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1689)
   [junit4]   2> 	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:857)
   [junit4]   2> 	... 8 more
   [junit4]   2> Caused by: org.apache.lucene.index.IndexNotFoundException: no segments* file found in MMapDirectory@/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MergeIndexesEmbeddedTest 7225193BA072C3CD-001/tempDir-004/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@3eba49df: files: [write.lock]
   [junit4]   2> 	at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:821)
   [junit4]   2> 	at org.apache.solr.update.SolrIndexWriter.<init>(SolrIndexWriter.java:78)
   [junit4]   2> 	at org.apache.solr.update.SolrIndexWriter.create(SolrIndexWriter.java:65)
   [junit4]   2> 	at org.apache.solr.update.DefaultSolrCoreState.createMainIndexWriter(DefaultSolrCoreState.java:273)
   [junit4]   2> 	at org.apache.solr.update.DefaultSolrCoreState.getIndexWriter(DefaultSolrCoreState.java:116)
   [junit4]   2> 	at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1546)
   [junit4]   2> 	... 10 more
   [junit4]   2> 
   [junit4]   2> 7220486 T21 oasu.SolrIndexWriter.close ERROR Error closing IndexWriter java.nio.channels.ClosedByInterruptException
   [junit4]   2> 	at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:202)
   [junit4]   2> 	at sun.nio.ch.FileChannelImpl.write(FileChannelImpl.java:210)
   [junit4]   2> 	at java.nio.channels.Channels.writeFullyImpl(Channels.java:78)
   [junit4]   2> 	at java.nio.channels.Channels.writeFully(Channels.java:101)
   [junit4]   2> 	at java.nio.channels.Channels.access$000(Channels.java:61)
   [junit4]   2> 	at java.nio.channels.Channels$1.write(Channels.java:174)
   [junit4]   2> 	at org.apache.lucene.store.FSDirectory$FSIndexOutput$1.write(FSDirectory.java:268)
   [junit4]   2> 	at java.util.zip.CheckedOutputStream.write(CheckedOutputStream.java:73)
   [junit4]   2> 	at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
   [junit4]   2> 	at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
   [junit4]   2> 	at org.apache.lucene.store.OutputStreamIndexOutput.getChecksum(OutputStreamIndexOutput.java:80)
   [junit4]   2> 	at org.apache.lucene.codecs.CodecUtil.writeFooter(CodecUtil.java:310)
   [junit4]   2> 	at org.apache.lucene.index.SegmentInfos.write(SegmentInfos.java:507)
   [junit4]   2> 	at org.apache.lucene.index.SegmentInfos.prepareCommit(SegmentInfos.java:731)
   [junit4]   2> 	at org.apache.lucene.index.IndexWriter.startCommit(IndexWriter.java:4290)
   [junit4]   2> 	at org.apache.lucene.index.IndexWriter.prepareCommitInternal(IndexWriter.java:2740)
   [junit4]   2> 	at org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:2843)
   [junit4]   2> 	at org.apache.lucene.index.IndexWriter.shutdown(IndexWriter.java:968)
   [junit4]   2> 	at org.apache.lucene.index.IndexWriter.close(IndexWriter.java:1011)
   [junit4]   2> 	at org.apache.solr.update.SolrIndexWriter.close(SolrIndexWriter.java:130)
   [junit4]   2> 	at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:539)
   [junit4]   2> 	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:796)
   [junit4]   2> 	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:652)
   [junit4]   2> 	at org.apache.solr.core.CoreContainer.create(CoreContainer.java:518)
   [junit4]   2> 	at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:283)
   [junit4]   2> 	at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:277)
   [junit4]   2> 	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 
   [junit4]   2> 7220490 T21 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 7220493 T21 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 7220495 T21 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 7220497 T21 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 7220499 T21 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 7220502 T21 oasc.PluginBag.createPlugin requestHandler : '/analysis/field' created with startup=lazy 
   [junit4]   2> 7220503 T21 oasc.RequestHandlers.initHandlersFromConfig Registered paths: /update/json,/schema,,/replication,/update/json/docs,/admin/logging,standard,/analysis/field,/admin/mbeans,/config,/admin/segments,/admin/file,/admin/properties,/update,/admin/ping,/get,/admin/system,/update/csv,/admin/threads,/admin/plugins,/admin/luke
   [junit4]   2> 7220503 T21 oasc.RequestHandlers.initHandlersFromConfig WARN no default request handler is registered (either '/select' or 'standard')
   [junit4]   2> 7220506 T21 oasc.SolrCore.initStatsCache Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 7220507 T21 oasu.UpdateHandler.<init> Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 7220507 T21 oasu.UpdateLog.init Initializing UpdateLog: dataDir=/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MergeIndexesEmbeddedTest 7225193BA072C3CD-001/tempDir-003 defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10
   [junit4]   2> 7220508 T21 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 7220508 T21 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 7220510 T21 oasc.CachingDirectoryFactory.closeCacheValue looking to close /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MergeIndexesEmbeddedTest 7225193BA072C3CD-001/tempDir-003/index [CachedDir<<refCount=0;path=/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MergeIndexesEmbeddedTest 7225193BA072C3CD-001/tempDir-003/index;done=true>>]
   [junit4]   2> 7220511 T21 oasc.CachingDirectoryFactory.close Closing directory: /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MergeIndexesEmbeddedTest 7225193BA072C3CD-001/tempDir-003/index
   [junit4]   2> 7220511 T21 oasc.SolrCore.close [core0]  CLOSING SolrCore org.apache.solr.core.SolrCore@56e70883
   [junit4]   2> 7220512 T21 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0,transaction_logs_total_size=0,transaction_logs_total_number=0}
   [junit4]   2> 7220512 T21 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
   [junit4]   2> 7220513 T21 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
   [junit4]   2> 7220513 T21 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
   [junit4]   2> 7220514 T21 oasc.SolrCore.closeSearcher [core0] Closing main searcher on request.
   [junit4]   2> 7220514 T21 oasc.CachingDirectoryFactory.close Closing StandardDirectoryFactory - 1 directories currently being tracked
   [junit4]   2> 7220515 T21 oasc.CachingDirectoryFactory.closeCacheValue looking to close /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MergeIndexesEmbeddedTest 7225193BA072C3CD-001/tempDir-003 [CachedDir<<refCount=0;path=/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MergeIndexesEmbeddedTest 7225193BA072C3CD-001/tempDir-003;done=false>>]
   [junit4]   2> 7220515 T21 oasc.CachingDirectoryFactory.close Closing directory: /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MergeIndexesEmbeddedTest 7225193BA072C3CD-001/tempDir-003
   [junit4]   2> 7220516 T21 oasc.CoreContainer.create ERROR Error creating core [core0]: Error opening new searcher org.apache.solr.common.SolrException: Error opening new searcher
   [junit4]   2> 	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:885)
   [junit4]   2> 	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:652)
   [junit4]   2> 	at org.apache.solr.core.CoreContainer.create(CoreContainer.java:518)
   [junit4]   2> 	at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:283)
   [junit4]   2> 	at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:277)
   [junit4]   2> 	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> Caused by: org.apache.solr.common.SolrException: Error opening new searcher
   [junit4]   2> 	at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1577)
   [junit4]   2> 	at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1689)
   [junit4]   2> 	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:857)
   [junit4]   2> 	... 8 more
   [junit4]   2> Caused by: org.apache.lucene.index.IndexNotFoundException: no segments* file found in MMapDirectory@/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MergeIndexesEmbeddedTest 7225193BA072C3CD-001/tempDir-003/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@3eba49df: files: [write.lock]
   [junit4]   2> 	at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:821)
   [junit4]   2> 	at org.apache.solr.update.SolrIndexWriter.<init>(SolrIndexWriter.java:78)
   [junit4]   2> 	at org.apache.solr.update.SolrIndexWriter.create(SolrIndexWriter.java:65)
   [junit4]   2> 	at org.apache.solr.update.DefaultSolrCoreState.createMainIndexWriter(DefaultSolrCoreState.java:273)
   [junit4]   2> 	at org.apache.solr.update.DefaultSolrCoreState.getIndexWriter(DefaultSolrCoreState.java:116)
   [junit4]   2> 	at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1546)
   [junit4]   2> 	... 10 more
   [junit4]   2> 
   [junit4]   2> 7220517 T14 oas.SolrTestCaseJ4.tearDown ###Ending testMergeIndexesByCoreName
   [junit4]   2> 7220518 T14 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=1859197274
   [junit4]   2> 7220529 T14 oas.SolrTestCaseJ4.setUp ###Starting testMergeIndexesByDirName
   [junit4]   2> 7220533 T14 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/solrj/src/test-files/solrj/solr/multicore/'
   [junit4]   2> 7220553 T14 oasc.SolrXmlConfig.fromFile Loading container configuration from /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/solrj/src/test-files/solrj/solr/multicore/solr.xml
   [junit4]   2> 7220572 T14 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/solrj/src/test-files/solrj/solr/multicore
   [junit4]   2> 7220573 T14 oasc.CoreContainer.<init> New CoreContainer 1741624376
   [junit4]   2> 7220573 T14 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/solrj/src/test-files/solrj/solr/multicore/]
   [junit4]   2> 7220574 T14 oasc.CoreContainer.load loading shared library: /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/solrj/src/test-files/solrj/solr/multicore/lib
   [junit4]   2> 7220574 T14 oasc.SolrResourceLoader.addToClassLoader WARN Can't find (or read) directory to add to classloader: lib (resolved as: /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/solrj/src/test-files/solrj/solr/multicore/lib).
   [junit4]   2> 7220585 T14 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 600000
   [junit4]   2> 7220586 T14 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: null
   [junit4]   2> 7220586 T14 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 60000
   [junit4]   2> 7220587 T14 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 7220587 T14 oashc.HttpShardHandlerFactory.getParameter Setting maxConnections to: 10000
   [junit4]   2> 7220588 T14 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 7220588 T14 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 7220589 T14 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 7220589 T14 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 7220590 T14 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 7220590 T14 oashc.HttpShardHandlerFactory.getParameter Setting useRetries to: false
   [junit4]   2> 7220591 T14 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=600000&connTimeout=60000&retry=true
   [junit4]   2> 7220592 T14 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 7220592 T14 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 7220593 T14 oasc.CoreContainer.load Node Name: null
   [junit4]   2> 7220594 T14 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/solrj/src/test-files/solrj/solr/multicore
   [junit4]   2> 7220595 T14 oasc.CoreDescriptor.<init> CORE DESCRIPTOR: {instanceDir=/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/solrj/src/test-files/solrj/solr/multicore/core1, dataDir=data/, config=solrconfig.xml, absoluteInstDir=/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/solrj/src/test-files/solrj/solr/multicore/core1/, loadOnStartup=true, name=core1, schema=schema.xml, transient=false}
   [junit4]   2> 7220596 T14 oasc.CorePropertiesLocator.discoverUnder Found core core1 in /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/solrj/src/test-files/solrj/solr/multicore/core1/
   [junit4]   2> 7220597 T14 oasc.CoreDescriptor.<init> CORE DESCRIPTOR: {dataDir=data/, instanceDir=/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/solrj/src/test-files/solrj/solr/multicore/core0, transient=false, schema=schema.xml, absoluteInstDir=/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/solrj/src/test-files/solrj/solr/multicore/core0/, name=core0, config=solrconfig.xml, loadOnStartup=true}
   [junit4]   2> 7220598 T14 oasc.CorePropertiesLocator.discoverUnder Found core core0 in /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/solrj/src/test-files/solrj/solr/multicore/core0/
   [junit4]   2> 7220599 T14 oasc.CorePropertiesLocator.discover Found 2 core definitions
   [junit4]   2> 7220600 T24 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/solrj/src/test-files/solrj/solr/multicore/core1/'
   [junit4]   2> 7220600 T25 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/solrj/src/test-files/solrj/solr/multicore/core0/'
   [junit4]   2> 7220621 T24 oasc.SolrConfig.refreshRequestParams current version of requestparams : -1
   [junit4]   2> 7220622 T25 oasc.SolrConfig.refreshRequestParams current version of requestparams : -1
   [junit4]   2> 7220635 T25 oasc.SolrConfig.<init> Using Lucene MatchVersion: 5.0.0
   [junit4]   2> 7220638 T24 oasc.SolrConfig.<init> Using Lucene MatchVersion: 5.0.0
   [junit4]   2> 7220663 T25 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 7220664 T25 oass.IndexSchema.readSchema Reading Solr Schema from /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/solrj/src/test-files/solrj/solr/multicore/core0/conf/schema.xml
   [junit4]   2> 7220665 T24 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 7220666 T24 oass.IndexSchema.readSchema Reading Solr Schema from /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/solrj/src/test-files/solrj/solr/multicore/core1/conf/schema.xml
   [junit4]   2> 7220671 T25 oass.IndexSchema.readSchema [core0] Schema name=example core zero
   [junit4]   2> 7220672 T24 oass.IndexSchema.readSchema [core1] Schema name=example core one
   [junit4]   2> 7220678 T25 oass.IndexSchema.readSchema default search field in schema is name
   [junit4]   2> 7220679 T25 oass.IndexSchema.readSchema query parser default operator is OR
   [junit4]   2> 7220680 T25 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 7220680 T24 oass.IndexSchema.readSchema default search field in schema is name
   [junit4]   2> 7220681 T25 oasc.CoreContainer.create Creating SolrCore 'core0' using configuration from instancedir /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/solrj/src/test-files/solrj/solr/multicore/core0/
   [junit4]   2> 7220681 T24 oass.IndexSchema.readSchema query parser default operator is OR
   [junit4]   2> 7220681 T25 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
   [junit4]   2> 7220682 T25 oasc.SolrCore.<init> [core0] Opening new SolrCore at /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/solrj/src/test-files/solrj/solr/multicore/core0/, dataDir=/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MergeIndexesEmbeddedTest 7225193BA072C3CD-001/tempDir-005/
   [junit4]   2> 7220682 T24 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 7220682 T25 oasc.SolrCore.<init> JMX monitoring not detected for core: core0
   [junit4]   2> 7220683 T24 oasc.CoreContainer.create Creating SolrCore 'core1' using configuration from instancedir /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/solrj/src/test-files/solrj/solr/multicore/core1/
   [junit4]   2> 7220683 T25 oasc.CachingDirectoryFactory.get return new directory for /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MergeIndexesEmbeddedTest 7225193BA072C3CD-001/tempDir-005
   [junit4]   2> 7220683 T24 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
   [junit4]   2> 7220684 T25 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MergeIndexesEmbeddedTest 7225193BA072C3CD-001/tempDir-005/index/
   [junit4]   2> 7220684 T24 oasc.SolrCore.<init> [core1] Opening new SolrCore at /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/solrj/src/test-files/solrj/solr/multicore/core1/, dataDir=/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MergeIndexesEmbeddedTest 7225193BA072C3CD-001/tempDir-006/
   [junit4]   2> 7220684 T25 oasc.SolrCore.initIndex WARN [core0] Solr index directory '/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MergeIndexesEmbeddedTest 7225193BA072C3CD-001/tempDir-005/index' doesn't exist. Creating new index...
   [junit4]   2> 7220684 T24 oasc.SolrCore.<init> JMX monitoring not detected for core: core1
   [junit4]   2> 7220685 T25 oasc.CachingDirectoryFactory.get return new directory for /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MergeIndexesEmbeddedTest 7225193BA072C3CD-001/tempDir-005/index
   [junit4]   2> 7220685 T24 oasc.CachingDirectoryFactory.get return new directory for /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MergeIndexesEmbeddedTest 7225193BA072C3CD-001/tempDir-006
   [junit4]   2> 7220686 T24 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MergeIndexesEmbeddedTest 7225193BA072C3CD-001/tempDir-006/index/
   [junit4]   2> 7220686 T24 oasc.SolrCore.initIndex WARN [core1] Solr index directory '/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MergeIndexesEmbeddedTest 7225193BA072C3CD-001/tempDir-006/index' doesn't exist. Creating new index...
   [junit4]   2> 7220687 T24 oasc.CachingDirectoryFactory.get return new directory for /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MergeIndexesEmbeddedTest 7225193BA072C3CD-001/tempDir-006/index
   [junit4]   2> 7220709 T24 oasu.SolrIndexWriter.close ERROR Error closing IndexWriter org.apache.lucene.util.ThreadInterruptedException: java.lang.InterruptedException: sleep interrupted
   [junit4]   2> 	at org.apache.lucene.util.IOUtils.fsync(IOUtils.java:405)
   [junit4]   2> 	at org.apache.lucene.store.FSDirectory.fsync(FSDirectory.java:278)
   [junit4]   2> 	at org.apache.lucene.store.FSDirectory.sync(FSDirectory.java:223)
   [junit4]   2> 	at org.apache.lucene.index.SegmentInfos.write(SegmentInfos.java:509)
   [junit4]   2> 	at org.apache.lucene.index.SegmentInfos.prepareCommit(SegmentInfos.java:731)
   [junit4]   2> 	at org.apache.lucene.index.IndexWriter.startCommit(IndexWriter.java:4290)
   [junit4]   2> 	at org.apache.lucene.index.IndexWriter.prepareCommitInternal(IndexWriter.java:2740)
   [junit4]   2> 	at org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:2843)
   [junit4]   2> 	at org.apache.lucene.index.IndexWriter.shutdown(IndexWriter.java:968)
   [junit4]   2> 	at org.apache.lucene.index.IndexWriter.close(IndexWriter.java:1011)
   [junit4]   2> 	at org.apache.solr.update.SolrIndexWriter.close(SolrIndexWriter.java:130)
   [junit4]   2> 	at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:539)
   [junit4]   2> 	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:796)
   [junit4]   2> 	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:652)
   [junit4]   2> 	at org.apache.solr.core.CoreContainer.create(CoreContainer.java:518)
   [junit4]   2> 	at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:283)
   [junit4]   2> 	at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:277)
   [junit4]   2> 	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 	Suppressed: java.nio.channels.ClosedByInterruptException
   [junit4]   2> 		at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:202)
   [junit4]   2> 		at sun.nio.ch.FileChannelImpl.force(FileChannelImpl.java:380)
   [junit4]   2> 		at org.apache.lucene.util.IOUtils.fsync(IOUtils.java:395)
   [junit4]   2> 		... 20 more
   [junit4]   2> Caused by: java.lang.InterruptedException: sleep interrupted
   [junit4]   2> 	at java.lang.Thread.sleep(Native Method)
   [junit4]   2> 	at org.apache.lucene.util.IOUtils.fsync(IOUtils.java:403)
   [junit4]   2> 	... 20 more
   [junit4]   2> 
   [junit4]   2> 7220709 T25 oasu.SolrIndexWriter.close ERROR Error closing IndexWriter org.apache.lucene.util.ThreadInterruptedException: java.lang.InterruptedException: sleep interrupted
   [junit4]   2> 	at org.apache.lucene.util.IOUtils.fsync(IOUtils.java:405)
   [junit4]   2> 	at org.apache.lucene.store.FSDirectory.renameFile(FSDirectory.java:233)
   [junit4]   2> 	at org.apache.lucene.index.SegmentInfos.finishCommit(SegmentInfos.java:773)
   [junit4]   2> 	at org.apache.lucene.index.IndexWriter.finishCommit(IndexWriter.java:2869)
   [junit4]   2> 	at org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:2850)
   [junit4]   2> 	at org.apache.lucene.index.IndexWriter.shutdown(IndexWriter.java:968)
   [junit4]   2> 	at org.apache.lucene.index.IndexWriter.close(IndexWriter.java:1011)
   [junit4]   2> 	at org.apache.solr.update.SolrIndexWriter.close(SolrIndexWriter.java:130)
   [junit4]   2> 	at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:539)
   [junit4]   2> 	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:796)
   [junit4]   2> 	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:652)
   [junit4]   2> 	at org.apache.solr.core.CoreContainer.create(CoreContainer.java:518)
   [junit4]   2> 	at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:283)
   [junit4]   2> 	at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:277)
   [junit4]   2> 	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 	Suppressed: java.nio.channels.ClosedByInterruptException
   [junit4]   2> 		at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:202)
   [junit4]   2> 		at sun.nio.ch.FileChannelImpl.force(FileChannelImpl.java:380)
   [junit4]   2> 		at org.apache.lucene.util.IOUtils.fsync(IOUtils.java:395)
   [junit4]   2> 		... 17 more
   [junit4]   2> Caused by: java.lang.InterruptedException: sleep interrupted
   [junit4]   2> 	at java.lang.Thread.sleep(Native Method)
   [junit4]   2> 	at org.apache.lucene.util.IOUtils.fsync(IOUtils.java:403)
   [junit4]   2> 	... 17 more
   [junit4]   2> 
   [junit4]   2> 7220712 T24 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 7220713 T25 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 7220715 T24 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 7220715 T25 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 7220716 T24 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 7220716 T25 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 7220718 T25 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 7220718 T24 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 7220719 T25 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 7220720 T24 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 7220722 T25 oasc.PluginBag.createPlugin requestHandler : '/analysis/field' created with startup=lazy 
   [junit4]   2> 7220722 T25 oasc.RequestHandlers.initHandlersFromConfig Registered paths: /update/json,/schema,,/replication,/update/json/docs,/admin/logging,standard,/analysis/field,/admin/mbeans,/config,/admin/segments,/admin/file,/admin/properties,/update,/admin/ping,/get,/admin/system,/update/csv,/admin/threads,/admin/plugins,/admin/luke
   [junit4]   2> 7220722 T25 oasc.RequestHandlers.initHandlersFromConfig WARN no default request handler is registered (either '/select' or 'standard')
   [junit4]   2> 7220724 T24 oasc.PluginBag.createPlugin requestHandler : '/analysis/field' created with startup=lazy 
   [junit4]   2> 7220724 T25 oasc.SolrCore.initStatsCache Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 7220724 T24 oasc.RequestHandlers.initHandlersFromConfig Registered paths: /update/json,/schema,,/replication,/update/json/docs,/admin/logging,standard,/analysis/field,/admin/mbeans,/config,/admin/segments,/admin/file,/admin/properties,/update,/a

[...truncated too long message...]

x
   [junit4]   2> 7220733 T25 oasc.CachingDirectoryFactory.closeCacheValue looking to close /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MergeIndexesEmbeddedTest 7225193BA072C3CD-001/tempDir-005 [CachedDir<<refCount=0;path=/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MergeIndexesEmbeddedTest 7225193BA072C3CD-001/tempDir-005;done=false>>]
   [junit4]   2> 7220734 T24 oasc.SolrCore.close [core1]  CLOSING SolrCore org.apache.solr.core.SolrCore@4ab0c9f0
   [junit4]   2> 7220734 T25 oasc.CachingDirectoryFactory.close Closing directory: /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MergeIndexesEmbeddedTest 7225193BA072C3CD-001/tempDir-005
   [junit4]   2> 7220735 T24 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0,transaction_logs_total_size=0,transaction_logs_total_number=0}
   [junit4]   2> 7220735 T25 oasc.CoreContainer.create ERROR Error creating core [core0]: Error opening new searcher org.apache.solr.common.SolrException: Error opening new searcher
   [junit4]   2> 	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:885)
   [junit4]   2> 	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:652)
   [junit4]   2> 	at org.apache.solr.core.CoreContainer.create(CoreContainer.java:518)
   [junit4]   2> 	at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:283)
   [junit4]   2> 	at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:277)
   [junit4]   2> 	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> Caused by: org.apache.solr.common.SolrException: Error opening new searcher
   [junit4]   2> 	at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1577)
   [junit4]   2> 	at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1689)
   [junit4]   2> 	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:857)
   [junit4]   2> 	... 8 more
   [junit4]   2> Caused by: org.apache.lucene.index.IndexNotFoundException: no segments* file found in MMapDirectory@/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MergeIndexesEmbeddedTest 7225193BA072C3CD-001/tempDir-005/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@3eba49df: files: [write.lock]
   [junit4]   2> 	at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:821)
   [junit4]   2> 	at org.apache.solr.update.SolrIndexWriter.<init>(SolrIndexWriter.java:78)
   [junit4]   2> 	at org.apache.solr.update.SolrIndexWriter.create(SolrIndexWriter.java:65)
   [junit4]   2> 	at org.apache.solr.update.DefaultSolrCoreState.createMainIndexWriter(DefaultSolrCoreState.java:273)
   [junit4]   2> 	at org.apache.solr.update.DefaultSolrCoreState.getIndexWriter(DefaultSolrCoreState.java:116)
   [junit4]   2> 	at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1546)
   [junit4]   2> 	... 10 more
   [junit4]   2> 
   [junit4]   2> 7220735 T24 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
   [junit4]   2> 7220737 T24 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
   [junit4]   2> 7220737 T24 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
   [junit4]   2> 7220738 T24 oasc.SolrCore.closeSearcher [core1] Closing main searcher on request.
   [junit4]   2> 7220738 T24 oasc.CachingDirectoryFactory.close Closing StandardDirectoryFactory - 1 directories currently being tracked
   [junit4]   2> 7220739 T24 oasc.CachingDirectoryFactory.closeCacheValue looking to close /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MergeIndexesEmbeddedTest 7225193BA072C3CD-001/tempDir-006 [CachedDir<<refCount=0;path=/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MergeIndexesEmbeddedTest 7225193BA072C3CD-001/tempDir-006;done=false>>]
   [junit4]   2> 7220739 T24 oasc.CachingDirectoryFactory.close Closing directory: /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MergeIndexesEmbeddedTest 7225193BA072C3CD-001/tempDir-006
   [junit4]   2> 7220740 T24 oasc.CoreContainer.create ERROR Error creating core [core1]: Error opening new searcher org.apache.solr.common.SolrException: Error opening new searcher
   [junit4]   2> 	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:885)
   [junit4]   2> 	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:652)
   [junit4]   2> 	at org.apache.solr.core.CoreContainer.create(CoreContainer.java:518)
   [junit4]   2> 	at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:283)
   [junit4]   2> 	at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:277)
   [junit4]   2> 	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> Caused by: org.apache.solr.common.SolrException: Error opening new searcher
   [junit4]   2> 	at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1577)
   [junit4]   2> 	at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1689)
   [junit4]   2> 	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:857)
   [junit4]   2> 	... 8 more
   [junit4]   2> Caused by: org.apache.lucene.index.IndexNotFoundException: no segments* file found in MMapDirectory@/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MergeIndexesEmbeddedTest 7225193BA072C3CD-001/tempDir-006/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@3eba49df: files: [write.lock]
   [junit4]   2> 	at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:821)
   [junit4]   2> 	at org.apache.solr.update.SolrIndexWriter.<init>(SolrIndexWriter.java:78)
   [junit4]   2> 	at org.apache.solr.update.SolrIndexWriter.create(SolrIndexWriter.java:65)
   [junit4]   2> 	at org.apache.solr.update.DefaultSolrCoreState.createMainIndexWriter(DefaultSolrCoreState.java:273)
   [junit4]   2> 	at org.apache.solr.update.DefaultSolrCoreState.getIndexWriter(DefaultSolrCoreState.java:116)
   [junit4]   2> 	at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1546)
   [junit4]   2> 	... 10 more
   [junit4]   2> 
   [junit4]   2> 7220741 T14 oas.SolrTestCaseJ4.tearDown ###Ending testMergeIndexesByDirName
   [junit4]   2> 7220741 T14 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=1741624376
   [junit4]   2> 7220745 T14 oas.SolrTestCaseJ4.deleteCore ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MergeIndexesEmbeddedTest 7225193BA072C3CD-001
   [junit4]   2> 7220579 T13 ccr.ThreadLeakControl.tryToInterruptAll All leaked threads terminated.
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene50): {}, docValues:{}, sim=DefaultSimilarity, locale=sk, timezone=SST
   [junit4]   2> NOTE: Linux 3.13.0-46-generic amd64/Oracle Corporation 1.7.0_80-ea (64-bit)/cpus=12,threads=1,free=347382136,total=525336576
   [junit4]   2> NOTE: All tests run in this JVM: [SolrQueryTest, MergeIndexesEmbeddedTest]
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=MergeIndexesEmbeddedTest -Dtests.seed=7225193BA072C3CD -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=sk -Dtests.timezone=SST -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR   0.00s J1 | MergeIndexesEmbeddedTest (suite) <<<
   [junit4]    > Throwable #1: java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([7225193BA072C3CD]:0)
   [junit4] Completed on J1 in 7220.85s, 1 test, 2 errors <<< FAILURES!

[...truncated 33 lines...]
BUILD FAILED
/home/jenkins/workspace/Lucene-Solr-5.x-Linux/build.xml:529: The following error occurred while executing this line:
/home/jenkins/workspace/Lucene-Solr-5.x-Linux/build.xml:477: The following error occurred while executing this line:
/home/jenkins/workspace/Lucene-Solr-5.x-Linux/build.xml:61: The following error occurred while executing this line:
/home/jenkins/workspace/Lucene-Solr-5.x-Linux/extra-targets.xml:39: The following error occurred while executing this line:
/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build.xml:204: The following error occurred while executing this line:
/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/common-build.xml:509: The following error occurred while executing this line:
/home/jenkins/workspace/Lucene-Solr-5.x-Linux/lucene/common-build.xml:1370: The following error occurred while executing this line:
/home/jenkins/workspace/Lucene-Solr-5.x-Linux/lucene/common-build.xml:976: There were test failures: 57 suites, 366 tests, 1 suite-level error, 1 error

Total time: 168 minutes 25 seconds
Build step 'Invoke Ant' marked build as failure
[description-setter] Description set: Java: 64bit/jdk1.7.0_80-ea-b05 -XX:+UseCompressedOops -XX:+UseG1GC
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-5.x-Linux (32bit/jdk1.8.0_40) - Build # 11980 - Still Failing!

Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-5.x-Linux/11980/
Java: 32bit/jdk1.8.0_40 -client -XX:+UseG1GC

1 tests failed.
FAILED:  org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest.test

Error Message:
There were too many update fails (21 > 20) - we expect it can happen, but shouldn't easily

Stack Trace:
java.lang.AssertionError: There were too many update fails (21 > 20) - we expect it can happen, but shouldn't easily
	at __randomizedtesting.SeedInfo.seed([E55FEF0060F5B0E:8601C12AA8F336F6]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.junit.Assert.assertFalse(Assert.java:68)
	at org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest.test(ChaosMonkeyNothingIsSafeTest.java:230)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:497)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:960)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:935)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 9907 lines...]
   [junit4] Suite: org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest
   [junit4]   2> Creating dataDir: /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest E55FEF0060F5B0E-001/init-core-data-001
   [junit4]   2> 590701 T4001 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
   [junit4]   2> 590703 T4001 oasc.ZkTestServer.run STARTING ZK TEST SERVER
   [junit4]   1> client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 590705 T4002 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
   [junit4]   2> 590804 T4001 oasc.ZkTestServer.run start zk server on port:40183
   [junit4]   2> 590805 T4001 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 590806 T4001 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 590808 T4009 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@14bcab3 name:ZooKeeperConnection Watcher:127.0.0.1:40183 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 590809 T4001 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 590809 T4001 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 590809 T4001 oascc.SolrZkClient.makePath makePath: /solr
   [junit4]   2> 590813 T4001 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 590814 T4001 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 590815 T4012 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@e545a3 name:ZooKeeperConnection Watcher:127.0.0.1:40183/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 590815 T4001 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 590816 T4001 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 590816 T4001 oascc.SolrZkClient.makePath makePath: /collections/collection1
   [junit4]   2> 590818 T4001 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
   [junit4]   2> 590819 T4001 oascc.SolrZkClient.makePath makePath: /collections/control_collection
   [junit4]   2> 590820 T4001 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
   [junit4]   2> 590823 T4001 oasc.AbstractZkTestCase.putConfig put /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 590824 T4001 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 590826 T4001 oasc.AbstractZkTestCase.putConfig put /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/core/src/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
   [junit4]   2> 590826 T4001 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
   [junit4]   2> 590828 T4001 oasc.AbstractZkTestCase.putConfig put /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 590828 T4001 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 590830 T4001 oasc.AbstractZkTestCase.putConfig put /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 590830 T4001 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 590832 T4001 oasc.AbstractZkTestCase.putConfig put /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 590832 T4001 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
   [junit4]   2> 590835 T4001 oasc.AbstractZkTestCase.putConfig put /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 590836 T4001 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
   [junit4]   2> 590837 T4001 oasc.AbstractZkTestCase.putConfig put /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 590837 T4001 oascc.SolrZkClient.makePath makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 590838 T4001 oasc.AbstractZkTestCase.putConfig put /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 590839 T4001 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 590842 T4001 oasc.AbstractZkTestCase.putConfig put /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 590843 T4001 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 590844 T4001 oasc.AbstractZkTestCase.putConfig put /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 590845 T4001 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 590846 T4001 oasc.AbstractZkTestCase.putConfig put /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 590846 T4001 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 590926 T4001 oas.SolrTestCaseJ4.writeCoreProperties Writing core.properties file to /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest E55FEF0060F5B0E-001/control-001/cores/collection1
   [junit4]   2> 590927 T4001 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 590930 T4001 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:47310
   [junit4]   2> 590931 T4001 oascse.JettySolrRunner$1.lifeCycleStarted Jetty properties: {solr.data.dir=/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest E55FEF0060F5B0E-001/tempDir-001/control/data, hostContext=/, hostPort=47310, coreRootDirectory=/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/../../../../../../../../../home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest E55FEF0060F5B0E-001/control-001/cores}
   [junit4]   2> 590931 T4001 oass.SolrDispatchFilter.init SolrDispatchFilter.init()sun.misc.Launcher$AppClassLoader@e2f2a
   [junit4]   2> 590932 T4001 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/../../../../../../../../../home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest E55FEF0060F5B0E-001/control-001/'
   [junit4]   2> 590951 T4001 oasc.SolrXmlConfig.fromFile Loading container configuration from /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/../../../../../../../../../home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest E55FEF0060F5B0E-001/control-001/solr.xml
   [junit4]   2> 590983 T4001 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/../../../../../../../../../home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest E55FEF0060F5B0E-001/control-001/cores
   [junit4]   2> 590984 T4001 oasc.CoreContainer.<init> New CoreContainer 27023842
   [junit4]   2> 590984 T4001 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/../../../../../../../../../home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest E55FEF0060F5B0E-001/control-001/]
   [junit4]   2> 590985 T4001 oasc.CoreContainer.load loading shared library: /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/../../../../../../../../../home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest E55FEF0060F5B0E-001/control-001/lib
   [junit4]   2> 590985 T4001 oasc.SolrResourceLoader.addToClassLoader WARN Can't find (or read) directory to add to classloader: lib (resolved as: /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/../../../../../../../../../home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest E55FEF0060F5B0E-001/control-001/lib).
   [junit4]   2> 590992 T4001 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 590993 T4001 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 590993 T4001 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 590993 T4001 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 590994 T4001 oashc.HttpShardHandlerFactory.getParameter Setting maxConnections to: 10000
   [junit4]   2> 590994 T4001 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 590995 T4001 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 590995 T4001 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 590995 T4001 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 590996 T4001 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 590996 T4001 oashc.HttpShardHandlerFactory.getParameter Setting useRetries to: false
   [junit4]   2> 590997 T4001 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 590998 T4001 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 590998 T4001 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 590998 T4001 oasc.CoreContainer.load Node Name: 127.0.0.1
   [junit4]   2> 590999 T4001 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:40183/solr
   [junit4]   2> 590999 T4001 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 591000 T4001 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 591001 T4001 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 591002 T4024 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1f6d716 name:ZooKeeperConnection Watcher:127.0.0.1:40183 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 591003 T4001 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 591003 T4001 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 591005 T4001 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 591007 T4027 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1ab1e54 name:ZooKeeperConnection Watcher:127.0.0.1:40183/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 591007 T4001 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 591009 T4001 oascc.SolrZkClient.makePath makePath: /overseer/queue
   [junit4]   2> 591010 T4001 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
   [junit4]   2> 591012 T4001 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
   [junit4]   2> 591014 T4001 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
   [junit4]   2> 591016 T4001 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
   [junit4]   2> 591018 T4001 oascc.SolrZkClient.makePath makePath: /live_nodes
   [junit4]   2> 591019 T4001 oascc.SolrZkClient.makePath makePath: /aliases.json
   [junit4]   2> 591021 T4001 oascc.SolrZkClient.makePath makePath: /clusterstate.json
   [junit4]   2> 591022 T4001 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:47310_
   [junit4]   2> 591023 T4001 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:47310_
   [junit4]   2> 591024 T4001 oascc.SolrZkClient.makePath makePath: /overseer_elect
   [junit4]   2> 591025 T4001 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
   [junit4]   2> 591027 T4001 oasc.Overseer.close Overseer (id=null) closing
   [junit4]   2> 591029 T4001 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:47310_
   [junit4]   2> 591029 T4001 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
   [junit4]   2> 591030 T4001 oasc.Overseer.start Overseer (id=93563877340610563-127.0.0.1:47310_-n_0000000000) starting
   [junit4]   2> 591031 T4001 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
   [junit4]   2> 591039 T4001 oasc.OverseerAutoReplicaFailoverThread.<init> Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 591039 T4029 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
   [junit4]   2> 591041 T4028 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
   [junit4]   2> 591039 T4001 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 591045 T4001 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/../../../../../../../../../home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest E55FEF0060F5B0E-001/control-001/cores
   [junit4]   2> 591046 T4001 oasc.CoreDescriptor.<init> CORE DESCRIPTOR: {name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, loadOnStartup=true, instanceDir=/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/../../../../../../../../../home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest E55FEF0060F5B0E-001/control-001/cores/collection1, collection=control_collection, absoluteInstDir=/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/../../../../../../../../../home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest E55FEF0060F5B0E-001/control-001/cores/collection1/, coreNodeName=, dataDir=data/, shard=}
   [junit4]   2> 591047 T4001 oasc.CorePropertiesLocator.discoverUnder Found core collection1 in /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/../../../../../../../../../home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest E55FEF0060F5B0E-001/control-001/cores/collection1/
   [junit4]   2> 591047 T4001 oasc.CorePropertiesLocator.discover Found 1 core definitions
   [junit4]   2> 591048 T4031 C:control_collection c:collection1 oasc.ZkController.publish publishing core=collection1 state=down collection=control_collection
   [junit4]   2> 591048 T4031 C:control_collection c:collection1 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 591049 T4027 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 591049 T4031 C:control_collection c:collection1 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 591049 T4028 oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"http://127.0.0.1:47310",
   [junit4]   2> 	  "node_name":"127.0.0.1:47310_",
   [junit4]   2> 	  "numShards":"1",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "collection":"control_collection",
   [junit4]   2> 	  "operation":"state"} current state version: 0
   [junit4]   2> 591050 T4028 oasco.ReplicaMutator.updateState Update state numShards=1 message={
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"http://127.0.0.1:47310",
   [junit4]   2> 	  "node_name":"127.0.0.1:47310_",
   [junit4]   2> 	  "numShards":"1",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "collection":"control_collection",
   [junit4]   2> 	  "operation":"state"}
   [junit4]   2> 591050 T4028 oasco.ClusterStateMutator.createCollection building a new cName: control_collection
   [junit4]   2> 591050 T4028 oasco.ReplicaMutator.updateState Assigning new node to shard shard=shard1
   [junit4]   2> 591054 T4027 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 592049 T4031 C:control_collection c:collection1 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
   [junit4]   2> 592050 T4031 C:control_collection c:collection1 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
   [junit4]   2> 592050 T4031 C:control_collection c:collection1 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 592051 T4031 C:control_collection c:collection1 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
   [junit4]   2> 592052 T4031 C:control_collection c:collection1 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 592053 T4031 C:control_collection c:collection1 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/../../../../../../../../../home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest E55FEF0060F5B0E-001/control-001/cores/collection1/'
   [junit4]   2> 592069 T4031 C:control_collection c:collection1 oasc.ZkController.watchZKConfDir watch zkdir /configs/conf1
   [junit4]   2> 592071 T4031 C:control_collection c:collection1 oasc.Config.<init> loaded config solrconfig.xml with version 0 
   [junit4]   2> 592077 T4031 C:control_collection c:collection1 oasc.SolrConfig.refreshRequestParams current version of requestparams : -1
   [junit4]   2> 592081 T4031 C:control_collection c:collection1 oasc.SolrConfig.<init> Using Lucene MatchVersion: 5.1.0
   [junit4]   2> 592091 T4031 C:control_collection c:collection1 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 592092 T4031 C:control_collection c:collection1 oass.IndexSchema.readSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 592097 T4031 C:control_collection c:collection1 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 592181 T4031 C:control_collection c:collection1 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 592182 T4031 C:control_collection c:collection1 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 592183 T4031 C:control_collection c:collection1 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 592186 T4031 C:control_collection c:collection1 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 592204 T4031 C:control_collection c:collection1 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 592205 T4031 C:control_collection c:collection1 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
   [junit4]   2> 592209 T4031 C:control_collection c:collection1 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/../../../../../../../../../home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest E55FEF0060F5B0E-001/control-001/cores/collection1/, dataDir=/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest E55FEF0060F5B0E-001/control-001/cores/collection1/data/
   [junit4]   2> 592209 T4031 C:control_collection c:collection1 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1db9550
   [junit4]   2> 592210 T4031 C:control_collection c:collection1 oasc.CachingDirectoryFactory.get return new directory for /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest E55FEF0060F5B0E-001/control-001/cores/collection1/data
   [junit4]   2> 592211 T4031 C:control_collection c:collection1 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest E55FEF0060F5B0E-001/control-001/cores/collection1/data/index/
   [junit4]   2> 592211 T4031 C:control_collection c:collection1 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest E55FEF0060F5B0E-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 592211 T4031 C:control_collection c:collection1 oasc.CachingDirectoryFactory.get return new directory for /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest E55FEF0060F5B0E-001/control-001/cores/collection1/data/index
   [junit4]   2> 592212 T4031 C:control_collection c:collection1 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=42, maxMergeAtOnceExplicit=32, maxMergedSegmentMB=2.060546875, floorSegmentMB=2.013671875, forceMergeDeletesPctAllowed=3.0332049209735623, segmentsPerTier=36.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.7639391077100658
   [junit4]   2> 592230 T4031 C:control_collection c:collection1 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest E55FEF0060F5B0E-001/control-001/cores/collection1/data/index,segFN=segments_1,generation=1}
   [junit4]   2> 592231 T4031 C:control_collection c:collection1 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 592236 T4031 C:control_collection c:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 592237 T4031 C:control_collection c:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 592237 T4031 C:control_collection c:collection1 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 592237 T4031 C:control_collection c:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 592237 T4031 C:control_collection c:collection1 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 592237 T4031 C:control_collection c:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 592238 T4031 C:control_collection c:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 592238 T4031 C:control_collection c:collection1 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 592238 T4031 C:control_collection c:collection1 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 592240 T4031 C:control_collection c:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 592242 T4031 C:control_collection c:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 592243 T4031 C:control_collection c:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 592244 T4031 C:control_collection c:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 592249 T4031 C:control_collection c:collection1 oasc.RequestHandlers.initHandlersFromConfig Registered paths: /admin/mbeans,standard,/update/csv,/update/json/docs,/admin/luke,/admin/segments,/get,/admin/system,/replication,/admin/properties,/config,/schema,/admin/plugins,/admin/logging,/update/json,/admin/threads,/admin/ping,/update,/admin/file
   [junit4]   2> 592250 T4031 C:control_collection c:collection1 oasc.SolrCore.initStatsCache Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 592251 T4031 C:control_collection c:collection1 oasu.UpdateHandler.<init> Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 592251 T4031 C:control_collection c:collection1 oasu.UpdateLog.init Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10
   [junit4]   2> 592251 T4031 C:control_collection c:collection1 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 592252 T4031 C:control_collection c:collection1 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 592252 T4031 C:control_collection c:collection1 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=23, maxMergeAtOnceExplicit=36, maxMergedSegmentMB=92.6708984375, floorSegmentMB=1.947265625, forceMergeDeletesPctAllowed=2.1133270538227777, segmentsPerTier=11.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 592253 T4031 C:control_collection c:collection1 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest E55FEF0060F5B0E-001/control-001/cores/collection1/data/index,segFN=segments_1,generation=1}
   [junit4]   2> 592254 T4031 C:control_collection c:collection1 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 592254 T4031 C:control_collection c:collection1 oass.SolrIndexSearcher.<init> Opening Searcher@1c1d63e[collection1] main
   [junit4]   2> 592254 T4031 C:control_collection c:collection1 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
   [junit4]   2> 592255 T4031 C:control_collection c:collection1 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 592255 T4031 C:control_collection c:collection1 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 592256 T4031 C:control_collection c:collection1 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 592256 T4031 C:control_collection c:collection1 oasr.RestManager.init Initializing RestManager with initArgs: {}
   [junit4]   2> 592256 T4031 C:control_collection c:collection1 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 592257 T4031 C:control_collection c:collection1 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 592257 T4031 C:control_collection c:collection1 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 592258 T4031 C:control_collection c:collection1 oasr.RestManager.init Initializing 0 registered ManagedResources
   [junit4]   2> 592258 T4031 C:control_collection c:collection1 oash.ReplicationHandler.inform Commits will be reserved for  10000
   [junit4]   2> 592259 T4032 C:control_collection c:collection1 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1c1d63e[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 592259 T4031 C:control_collection c:collection1 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 592260 T4035 C:control_collection S:shard1 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:47310 collection:control_collection shard:shard1
   [junit4]   2> 592261 T4001 oass.SolrDispatchFilter.init user.dir=/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0
   [junit4]   2> 592262 T4001 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 592261 T4035 C:control_collection S:shard1 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 592262 T4001 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 592263 T4001 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 592265 T4038 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7347b7 name:ZooKeeperConnection Watcher:127.0.0.1:40183/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 592265 T4001 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 592266 T4001 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 592266 T4001 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 592267 T4035 C:control_collection S:shard1 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
   [junit4]   2> 592269 T4027 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 592269 T4035 C:control_collection S:shard1 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
   [junit4]   2> 592269 T4035 C:control_collection S:shard1 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
   [junit4]   2> ASYNC  NEW_CORE C2057 name=collection1 org.apache.solr.core.SolrCore@1992794 url=http://127.0.0.1:47310/collection1 node=127.0.0.1:47310_ C2057_STATE=coll:control_collection core:collection1 props:{core=collection1, base_url=http://127.0.0.1:47310, node_name=127.0.0.1:47310_, state=down}
   [junit4]   2> 592270 T4035 C:control_collection S:shard1 C2057 P47310 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:47310/collection1/
   [junit4]   2> 592270 T4001 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 592270 T4035 C:control_collection S:shard1 C2057 P47310 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
   [junit4]   2> 592270 T4035 C:control_collection S:shard1 C2057 P47310 oasc.SyncStrategy.syncToMe http://127.0.0.1:47310/collection1/ has no replicas
   [junit4]   2> 592271 T4035 C:control_collection S:shard1 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:47310/collection1/ shard1
   [junit4]   2> 592271 T4035 C:control_collection S:shard1 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 592276 T4028 oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "operation":"leader",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "collection":"control_collection"} current state version: 1
   [junit4]   2> 592279 T4027 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 592279 T4028 oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "operation":"leader",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "collection":"control_collection",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:47310",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "state":"active"} current state version: 1
   [junit4]   2> 592341 T4001 oas.SolrTestCaseJ4.writeCoreProperties Writing core.properties file to /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest E55FEF0060F5B0E-001/shard-1-001/cores/collection1
   [junit4]   2> 592342 T4001 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 1 in directory /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest E55FEF0060F5B0E-001/shard-1-001
   [junit4]   2> 592343 T4001 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 592345 T4001 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:60813
   [junit4]   2> 592345 T4001 oascse.JettySolrRunner$1.lifeCycleStarted Jetty properties: {solr.data.dir=/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest E55FEF0060F5B0E-001/tempDir-001/jetty1, solrconfig=solrconfig.xml, hostContext=/, hostPort=60813, coreRootDirectory=/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest E55FEF0060F5B0E-001/shard-1-001/cores}
   [junit4]   2> 592346 T4001 oass.SolrDispatchFilter.init SolrDispatchFilter.init()sun.misc.Launcher$AppClassLoader@e2f2a
   [junit4]   2> 592346 T4001 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest E55FEF0060F5B0E-001/shard-1-001/'
   [junit4]   2> 592362 T4001 oasc.SolrXmlConfig.fromFile Loading container configuration from /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest E55FEF0060F5B0E-001/shard-1-001/solr.xml
   [junit4]   2> 592367 T4001 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest E55FEF0060F5B0E-001/shard-1-001/cores
   [junit4]   2> 592367 T4001 oasc.CoreContainer.<init> New CoreContainer 13353577
   [junit4]   2> 592367 T4001 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest E55FEF0060F5B0E-001/shard-1-001/]
   [junit4]   2> 592368 T4001 oasc.CoreContainer.load loading shared library: /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest E55FEF0060F5B0E-001/shard-1-001/lib
   [junit4]   2> 592368 T4001 oasc.SolrResourceLoader.addToClassLoader WARN Can't find (or read) directory to add to classloader: lib (resolved as: /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest E55FEF0060F5B0E-001/shard-1-001/lib).
   [junit4]   2> 592375 T4001 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 592375 T4001 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 592376 T4001 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 592376 T4001 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 592376 T4001 oashc.HttpShardHandlerFactory.getParameter Setting maxConnections to: 10000
   [junit4]   2> 592376 T4001 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 592377 T4001 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 592377 T4001 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 592377 T4001 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 592377 T4001 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 592378 T4001 oashc.HttpShardHandlerFactory.getParameter Setting useRetries to: false
   [junit4]   2> 592378 T4001 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 592378 T4001 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 592379 T4001 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 592379 T4001 oasc.CoreContainer.load Node Name: 127.0.0.1
   [junit4]   2> 592379 T4001 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:40183/solr
   [junit4]   2> 592379 T4001 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 592380 T4001 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 592380 T4001 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 592381 T4027 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 592382 T4038 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 592383 T4050 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@144e51d name:ZooKeeperConnection Watcher:127.0.0.1:40183 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 592383 T4001 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 592384 T4001 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 592386 T4001 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 592387 T4053 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@895bfa name:ZooKeeperConnection Watcher:127.0.0.1:40183/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 592388 T4001 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 592391 T4001 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 592429 T4035 C:control_collection S:shard1 oasc.ZkController.register We are http://127.0.0.1:47310/collection1/ and leader is http://127.0.0.1:47310/collection1/
   [junit4]   2> 592430 T4035 C:control_collection S:shard1 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:47310
   [junit4]   2> 592430 T4035 C:control_collection S:shard1 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
   [junit4]   2> 592430 T4035 C:control_collection S:shard1 c:collection1 oasc.ZkController.publish publishing core=collection1 state=active collection=control_collection
   [junit4]   2> 592430 T4035 C:control_collection S:shard1 c:collection1 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 592431 T4027 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 592433 T4028 oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "core_node_name":"core_node1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"http://127.0.0.1:47310",
   [junit4]   2> 	  "node_name":"127.0.0.1:47310_",
   [junit4]   2> 	  "numShards":"1",
   [junit4]   2> 	  "state":"active",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "collection":"control_collection",
   [junit4]   2> 	  "operation":"state"} current state version: 2
   [junit4]   2> 592433 T4028 oasco.ReplicaMutator.updateState Update state numShards=1 message={
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "core_node_name":"core_node1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"http://127.0.0.1:47310",
   [junit4]   2> 	  "node_name":"127.0.0.1:47310_",
   [junit4]   2> 	  "numShards":"1",
   [junit4]   2> 	  "state":"active",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "collection":"control_collection",
   [junit4]   2> 	  "operation":"state"}
   [junit4]   2> 592536 T4027 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 592536 T4053 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 592536 T4038 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 593394 T4001 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:60813_
   [junit4]   2> 593395 T4001 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:60813_
   [junit4]   2> 593398 T4001 oasc.Overseer.close Overseer (id=null) closing
   [junit4]   2> 593400 T4001 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest E55FEF0060F5B0E-001/shard-1-001/cores
   [junit4]   2> 593401 T4001 oasc.CoreDescriptor.<init> CORE DESCRIPTOR: {name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, loadOnStartup=true, instanceDir=/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest E55FEF0060F5B0E-001/shard-1-001/cores/collection1, collection=collection1, absoluteInstDir=/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest E55FEF0060F5B0E-001/shard-1-001/cores/collection1/, coreNodeName=, dataDir=data/, shard=}
   [junit4]   2> 593401 T4001 oasc.CorePropertiesLocator.discoverUnder Found core collection1 in /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest E55FEF0060F5B0E-001/shard-1-001/cores/collection1/
   [junit4]   2> 593401 T4001 oasc.CorePropertiesLocator.discover Found 1 core definitions
   [junit4]   2> 593402 T4054 C:collection1 c:collection1 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
   [junit4]   2> 593402 T4054 C:collection1 c:collection1 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 593403 T4027 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 593403 T4054 C:collection1 c:collection1 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 593403 T4028 oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"http://127.0.0.1:60813",
   [junit4]   2> 	  "node_name":"127.0.0.1:60813_",
   [junit4]   2> 	  "numShards":"1",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "operation":"state"} current state version: 3
   [junit4]   2> 593404 T4028 oasco.ReplicaMutator.updateState Update state numShards=1 message={
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"http://127.0.0.1:60813",
   [junit4]   2> 	  "node_name":"127.0.0.1:60813_",
   [junit4]   2> 	  "numShards":"1",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "operation":"state"}
   [junit4]   2> 593404 T4028 oasco.ClusterStateMutator.createCollection building a new cName: collection1
   [junit4]   2> 593404 T4028 oasco.ReplicaMutator.updateState Assigning new node to shard shard=shard1
   [junit4]   2> 593506 T4053 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 593506 T4038 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 593506 T4027 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 594403 T4054 C:collection1 c:collection1 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
   [junit4]   2> 594404 T4054 C:collection1 c:collection1 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
   [junit4]   2> 594405 T4054 C:collection1 c:collection1 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 594405 T4054 C:collection1 c:collection1 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
   [junit4]   2> 594405 T4054 C:collection1 c:collection1 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 594406 T4054 C:collection1 c:collection1 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest E55FEF0060F5B0E-001/shard-1-001/cores/collection1/'
   [junit4]   2> 594424 T4054 C:collection1 c:collection1 oasc.ZkController.watchZKConfDir watch zkdir /configs/conf1
   [junit4]   2> 594426 T4054 C:collection1 c:collection1 oasc.Config.<init> loaded config solrconfig.xml with version 0 
   [junit4]   2> 594431 T4054 C:collection1 c:collection1 oasc.SolrConfig.refreshRequestParams current version of requestparams : -1
   [junit4]   2> 594437 T4054 C:collection1 c:collection1 oasc.SolrConfig.<init> Using Lucene MatchVersion: 5.1.0
   [junit4]   2> 594446 T4054 C:collection1 c:collection1 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 594447 T4054 C:collection1 c:collection1 oass.IndexSchema.readSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 594452 T4054 C:collection1 c:collection1 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 594569 T4054 C:collection1 c:collection1 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 594570 T4054 C:collection1 c:collection1 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 594571 T4054 C:collection1 c:collection1 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 594573 T4054 C:collection1 c:collection1 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 594589 T4054 C:collection1 c:collection1 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection collection1
   [junit4]   2> 594589 T4054 C:collection1 c:collection1 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
   [junit4]   2> 594589 T4054 C:collection1 c:collection1 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest E55FEF0060F5B0E-001/shard-1-001/cores/collection1/, dataDir=/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest E55FEF0060F5B0E-001/shard-1-001/cores/collection1/data/
   [junit4]   2> 594590 T4054 C:collection1 c:collection1 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1db9550
   [junit4]   2> 594590 T4054 C:collection1 c:collection1 oasc.CachingDirectoryFactory.get return new directory for /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest E55FEF0060F5B0E-001/shard-1-001/cores/collection1/data
   [junit4]   2> 594591 T4054 C:collection1 c:collection1 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest E55FEF0060F5B0E-001/shard-1-001/cores/collection1/data/index/
   [junit4]   2> 594591 T4054 C:collection1 c:collection1 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest E55FEF0060F5B0E-001/shard-1-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 594591 T4054 C:collection1 c:collection1 oasc.CachingDirectoryFactory.get return new directory for /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest E55FEF0060F5B0E-001/shard-1-001/cores/collection1/data/index
   [junit4]   2> 594592 T4054 C:collection1 c:collection1 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=42, maxMergeAtOnceExplicit=32, maxMergedSegmentMB=2.060546875, floorSegmentMB=2.013671875, forceMergeDeletesPctAllowed=3.0332049209735623, segmentsPerTier=36.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.7639391077100658
   [junit4]   2> 594621 T4054 C:collection1 c:collection1 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest E55FEF0060F5B0E-001/shard-1-001/cores/collection1/data/index,segFN=segments_1,generation=1}
   [junit4]   2> 594621 T4054 C:collection1 c:collection1 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 594628 T4054 C:collection1 c:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 594629 T4054 C:collection1 c:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 594629 T4054 C:collection1 c:collection1 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 594630 T4054 C:collection1 c:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 594630 T4054 C:collection1 c:collection1 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 594630 T4054 C:collection1 c:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 594631 T4054 C:collection1 c:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 594631 T4054 C:collection1 c:collection1 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 594631 T4054 C:collection1 c:collection1 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 594634 T4054 C:collection1 c:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 594636 T4054 C:collection1 c:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 594638 T4054 C:collection1 c:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 594641 T4054 C:collection1 c:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 594653 T4054 C:collection1 c:collection1 oasc.RequestHandlers.initHandlersFromConfig Registered paths: /admin/mbeans,standard,/update/csv,/update/json/docs,/admin/luke,/admin/segments,/get,/admin/system,/replication,/admin/properties,/config,/schema,/admin/plugins,/admin/logging,/update/json,/admin/threads,/admin/ping,/update,/admin/file
   [junit4]   2> 594654 T4054 C:collection1 c:collection1 oasc.SolrCore.initStatsCache Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 594655 T4054 C:collection1 c:collection1 oasu.UpdateHandler.<init> Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 594655 T4054 C:collection1 c:collection1 oasu.UpdateLog.init Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10
   [junit4]   2> 594655 T4054 C:collection1 c:collection1 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 594656 T4054 C:collection1 c:collection1 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 594658 T4054 C:collection1 c:collection1 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=23, maxMergeAtOnceExplicit=36, maxMergedSegmentMB=92.6708984375, floorSegmentMB=1.947265625, forceMergeDeletesPctAllowed=2.1133270538227777, segmentsPerTier=11.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 594659 T4054 C:collection1 c:collection1 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest E55FEF0060F5B0E-001/shard-1-001/cores/collection1/data/index,segFN=segments_1,generation=1}
   [junit4]   2> 594659 T4054 C:collection1 c:collection1 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 594659 T4054 C:collection1 c:collection1 oass.SolrIndexSearcher.<init> Opening Searcher@1cd52be[collection1] main
   [junit4]   2> 594660 T4054 C:collection1 c:collection1 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
   [junit4]   2> 594660 T4054 C:collection1 c:collection1 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 594661 T4054 C:collection1 c:collection1 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 594661 T4054 C:collection1 c:collection1 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 594661 T4054 C:collection1 c:collection1 oasr.RestManager.init Initializing RestManager with initArgs: {}
   [junit4]   2> 594661 T4054 C:collection1 c:collection1 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 594662 T4054 C:collection1 c:collection1 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 594662 T4054 C:collection1 c:collection1 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 594663 T4054 C:collection1 c:collection1 oasr.RestManager.init Initializing 0 registered ManagedResources
   [junit4]   2> 594663 T4054 C:collection1 c:collection1 oash.ReplicationHandler.inform Commits will be reserved for  10000
   [junit4]   2> 594664 T4055 C:collection1 c:collection1 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1cd52be[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 594665 T4054 C:collection1 c:collection1 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 594666 T4058 C:collection1 S:shard1 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:60813 collection:collection1 shard:shard1
   [junit4]   2> 594666 T4001 oass.SolrDispatchFilter.init user.dir=/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0
   [junit4]   2> 594666 T4058 C:collection1 S:shard1 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
   [junit4]   2> 594667 T4001 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 594669 T4058 C:collection1 S:shard1 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
   [junit4]   2> 594670 T4027 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 594670 T4058 C:collection1 S:shard1 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
   [junit4]   2> 594671 T4058 C:collection1 S:shard1 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
   [junit4]   2> 594671 T4028 oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "operation":"leader",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "collection":"collection1"} current state version: 4
   [junit4]   2> ASYNC  NEW_CORE C2058 name=collection1 org.apache.solr.core.SolrCore@117d0cb url=http://127.0.0.1:60813/collection1 node=127.0.0.1:60813_ C2058_STATE=coll:collection1 core:collection1 props:{core=collection1, base_url=http://127.0.0.1:60813, node_name=127.0.0.1:60813_, state=down}
   [junit4]   2> 594671 T4058 C:collection1 S:shard1 C2058 P60813 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:60813/collection1/
   [junit4]   2> 594671 T4058 C:collection1 S:shard1 C2058 P60813 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
   [junit4]   2> 594672 T4058 C:collection1 S:shard1 C2058 P60813 oasc.SyncStrategy.syncToMe http://127.0.0.1:60813/collection1/ has no replicas
   [junit4]   2> 594672 T4058 C:collection1 S:shard1 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:60813/collection1/ shard1
   [junit4]   2> 594672 T4058 C:collection1 S:shard1 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
   [junit4]   2> 594674 T4027 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 594675 T4028 oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "operation":"leader",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:60813",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "state":"active"} current state version: 4
   [junit4]   2> 594775 T4001 oas.SolrTestCaseJ4.writeCoreProperties Writing core.properties file to /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest E55FEF0060F5B0E-001/shard-2-001/cores/collection1
   [junit4]   2> 594776 T4001 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 2 in directory /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest E55FEF0060F5B0E-001/shard-2-001
   [junit4]   2> 594777 T4053 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 594777 T4027 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 594777 T4038 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 594777 T4001 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 594782 T4001 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:48846
   [junit4]   2> 594783 T4001 oascse.JettySolrRunner$1.lifeCycleStarted Jetty properties: {solr.data.dir=/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest E55FEF0060F5B0E-001/tempDir-001/jetty2, solrconfig=solrconfig.xml, hostContext=/, hostPort=48846, coreRootDirectory=/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest E55FEF0060F5B0E-001/shard-2-001/cores}
   [junit4]   2> 594784 T4001 oass.SolrDispatchFilter.init SolrDispatchFilter.init()sun.misc.Launcher$AppClassLoader@e2f2a
   [junit4]   2> 594784 T4001 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest E55FEF0060F5B0E-001/shard-2-001/'
   [junit4]   2> 594803 T4001 oasc.SolrXmlConfig.fromFile Loading container configuration from /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest E55FEF0060F5B0E-001/shard-2-001/solr.xml
   [junit4]   2> 594808 T4001 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest E55FEF0060F5B0E-001/shard-2-001/cores
   [junit4]   2> 594809 T4001 oasc.CoreContainer.<init> New CoreContainer 20037208
   [junit4]   2> 594809 T4001 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest E55FEF0060F5B0E-001/shard-2-001/]
   [junit4]   2> 594810 T4001 oasc.CoreContainer.load loading shared library: /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest E55FEF0060F5B0E-001/shard-2-001/lib
   [junit4]   2> 594810 T4001 oasc.SolrResourceLoader.addToClassLoader WARN Can't find (or read) directory to add to classloader: lib (resolved as: /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest E55FEF0060F5B0E-001/shard-2-001/lib).
   [junit4]   2> 594821 T4001 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 594822 T4001 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 594822 T4001 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 594822 T4001 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 594822 T4001 oashc.HttpShardHandlerFactory.getParameter Setting maxConnections to: 10000
   [junit4]   2> 594823 T4001 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 594823 T4001 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 594823 T4001 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 594823 T4001 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 594824 T4001 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 594824 T4001 oashc.HttpShardHandlerFactory.getParameter Setting useRetries to: false
   [junit4]   2> 594825 T4001 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 594825 T4058 C:collection1 S:shard1 oasc.ZkController.register We are http://127.0.0.1:60813/collection1/ and leader is http://127.0.0.1:60813/collection1/
   [junit4]   2> 594825 T4001 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 594825 T4058 C:collection1 S:shard1 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:60813
   [junit4]   2> 594826 T4058 C:collection1 S:shard1 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
   [junit4]   2> 594826 T4001 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 594826 T4058 C:collection1 S:shard1 c:collection1 oasc.ZkController.publish publishing core=collection1 state=active collection=collection1
   [junit4]   2> 594826 T4058 C:collection1 S:shard1 c:collection1 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 594827 T4001 oasc.CoreContainer.load Node Name: 127.0.0.1
   [junit4]   2> 594827 T4001 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:40183/solr
   [junit4]   2> 594827 T4001 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 594827 T4001 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 594829 T4001 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 594829 T4027 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 594831 T4070 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@a339f5 name:ZooKeeperConnection Watcher:127.0.0.1:40183 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 594831 T4001 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 594832 T4001 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 594833 T4028 oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "core_node_name":"core_node1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"http://127.0.0.1:60813",
   [junit4]   2> 	  "node_name":"127.0.0.1:60813_",
   [junit4]   2> 	  "numShards":"1",
   [junit4]   2> 	  "state":"active",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "operation":"state"} current state version: 5
   [junit4]   2> 594834 T4028 oasco.ReplicaMutator.updateState Update state numShards=1 message={
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "core_node_name":"core_node1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"http://127.0.0.1:60813",
   [junit4]   2> 	  "node_name":"127.0.0.1:60813_",
   [junit4]   2> 	  "numShards":"1",
   [junit4]   2> 	  "state":"active",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "operation":"state"}
   [junit4]   2> 594834 T4001 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 594836 T4073 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@ff1313 name:ZooKeeperConnection Watcher:127.0.0.1:40183/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 594837 T4001 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 594842 T4001 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 594937 T4053 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 594937 T4073 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 594937 T4038 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 594937 T4027 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 595845 T4001 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:48846_
   [junit4]   2> 595846 T4001 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:48846_
   [junit4]   2> 595848 T4001 oasc.Overseer.close Overseer (id=null) closing
   [junit4]   2> 595852 T4001 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest E55FEF0060F5B0E-001/shard-2-001/cores
   [junit4]   2> 595853 T4001 oasc.CoreDescriptor.<init> CORE DESCRIPTOR: {name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, loadOnStartup=true, instanceDir=/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest E55FEF0060F5B0E-001/shard-2-001/cores/collection1, collection=collection1, absoluteInstDir=/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest E55FEF0060F5B0E-001/shard-2-001/cores/collection1/, coreNodeName=, dataDir=data/, shard=}
   [junit4]   2> 595853 T4001 oasc.CorePropertiesLocator.discoverUnder Found core collection1 in /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest E55FEF0060F5B0E-001/shard-2-001/cores/collection1/
   [junit4]   2> 595854 T4001 oasc.CorePropertiesLocator.discover Found 1 core definitions
   [junit4]   2> 595854 T4074 C:collection1 c:collection1 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
   [junit4]   2> 595854 T4074 C:collection1 c:collection1 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 595855 T4027 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 595855 T4074 C:collection1 c:collection1 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 595856 T4028 oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"http://127.0.0.1:48846",
   [junit4]   2> 	  "node_name":"127.0.0.1:48846_",
   [junit4]   2> 	  "numShards":"1",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "operation":"state"} current state version: 6
   [junit4]   2> 595857 T4028 oasco.ReplicaMutator.updateState Update state numShards=1 message={
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"http://127.0.0.1:48846",
   [junit4]   2> 	  "node_name":"127.0.0.1:48846_",
   [junit4]   2> 	  "numShards":"1",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "operation":"state"}
   [junit4]   2> 595857 T4028 oasco.ReplicaMutator.updateState Collection already exists with numShards=1
   [junit4]   2> 595857 T4028 oasco.ReplicaMutator.updateState Assigning new node to shard shard=shard1
   [junit4]   2> 595960 T4053 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 595960 T4027 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 595960 T4038 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 595960 T4073 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 596856 T4074 C:collection1 c:collection1 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
   [junit4]   2> 596856 T4074 C:collection1 c:collection1 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
   [junit4]   2> 596857 T4074 C:collection1 c:collection1 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 596857 T4074 C:collection1 c:collection1 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
   [junit4]   2> 596858 T4074 C:collection1 c:collection1 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 596858 T4074 C:collection1 c:collection1 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest E55FEF0060F5B0E-001/shard-2-001/cores/collection1/'
   [junit4]   2> 596875 T4074 C:collection1 c:collection1 oasc.ZkController.watchZKConfDir watch zkdir /configs/conf1
   [junit4]   2> 596877 T4074 C:collection1 c:collection1 oasc.Config.<init> loaded config solrconfig.xml with version 0 
   [junit4]   2> 596882 T4074 C:collection1 c:collection1 oasc.SolrConfig.refreshRequestParams current version of requestparams : -1
   [junit4]   2> 596887 T4074 C:collection1 c:collection1 oasc.SolrConfig.<init> Using Lucene MatchVersion: 5.1.0
   [junit4]   2> 596897 T4074 C:collection1 c:collection1 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 596898 T4074 C:collection1 c:collection1 oass.IndexSchema.readSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 596904 T4074 C:collection1 c:collection1 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 596979 T4074 C:collection1 c:collection1 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 596981 T4074 C:collection1 c:collection1 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 596982 T4074 C:collection1 c:collection1 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 596984 T4074 C:collection1 c:collection1 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 597000 T4074 C:collection1 c:collection1 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection collection1
   [junit4]   2> 597001 T4074 C:collection1 c:collection1 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
   [junit4]   2> 597001 T4074 C:collection1 c:collection1 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest E55FEF0060F5B0E-001/shard-2-001/cores/collection1/, dataDir=/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest E55FEF0060F5B0E-001/shard-2-001/cores/collection1/data/
   [junit4]   2> 597002 T4074 C:collection1 c:collection1 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1db9550
   [junit4]   2> 597002 T4074 C:collection1 c:collection1 oasc.CachingDirectoryFactory.get return new directory for /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest E55FEF0060F5B0E-001/shard-2-001/cores/collection1/data
   [junit4]   2> 597003 T4074 C:collection1 c:collection1 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest E55FEF0060F5B0E-001/shard-2-001/cores/collection1/data/index/
   [junit4]   2> 597003 T4074 C:collection1 c:collection1 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest E55FEF0060F5B0E-001/shard-2-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 597004 T4074 C:collection1 c:collection1 oasc.CachingDirectoryFactory.get return new directory for /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest E55FEF0060F5B0E-001/shard-2-001/cores/collection1/data/index
   [junit4]   2> 597004 T4074 C:collection1 c:collection1 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=42, maxMergeAtOnceExplicit=32, maxMergedSegmentMB=2.060546875, floorSegmentMB=2.013671875, forceMergeDeletesPctAllowed=3.0332049209735623, segmentsPerTier=36.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.7639391077100658
   [junit4]   2> 597037 T4074 C:collection1 c:collection1 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest E55FEF0060F5B0E-001/shard-2-001/cores/collection1/data/index,segFN=segments_1,generation=1}
   [junit4]   2> 597037 T4074 C:collection1 c:collection1 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 597042 T4074 C:collection1 c:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 597042 T4074 C:collection1 c:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 597043 T4074 C:collection1 c:collection1 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 597043 T4074 C:collection1 c:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 597043 T4074 C:collection1 c:collection1 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 597043 T4074 C:collection1 c:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 597043 T4074 C:collection1 c:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 597044 T4074 C:collection1 c:collection1 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 597045 T4074 C:collection1 c:collection1 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 597046 T4074 C:collection1 c:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 597048 T4074 C:collection1 c:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 597049 T4074 C:collection1 c:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 597051 T4074 C:collection1 c:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 597061 T4074 C:collection1 c:collection1 oasc.RequestHandlers.initHandlersFromConfig Registered paths: /admin/mbeans,standard,/update/csv,/update/json/docs,/admin/luke,/admin/segments,/get,/admin/system,/replication,/admin/properties,/config,/schema,/admin/plugins,/admin/logging,/update/json,/admin/threads,/admin/ping,/update,/admin/file
   [junit4]   2> 597062 T4074 C:collection1 c:collection1 oasc.SolrCore.initStatsCache Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 597063 T4074 C:collection1 c:collection1 oasu.UpdateHandler.<init> Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 597063 T4074 C:collection1 c:collection1 oasu.UpdateLog.init Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10
   [junit4]   2> 597063 T4074 C:collection1 c:collection1 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 597063 T4074 C:collection1 c:collection1 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 597067 T4074 C:collection1 c:collection1 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=23, maxMergeAtOnceExplicit=36, maxMergedSegmentMB=92.6708984375, floorSegmentMB=1.947265625, forceMergeDeletesPctAllowed=2.1133270538227777, segmentsPerTier=11.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 597068 T4074 C:collection1 c:collection1 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-

[...truncated too long message...]

er  a listener was removed because of core close
   [junit4]   2> 627022 T4112 oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "core_node_name":"core_node2",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"http://127.0.0.1:48846",
   [junit4]   2> 	  "node_name":"127.0.0.1:48846_",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "operation":"state"} current state version: 13
   [junit4]   2> 627022 T4001 C:control_collection S:shard1 c:collection1 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=2,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=755,cumulative_deletesById=372,cumulative_deletesByQuery=1,cumulative_errors=0,transaction_logs_total_size=87357,transaction_logs_total_number=1}
   [junit4]   2> 627023 T4112 oasco.ReplicaMutator.updateState Update state numShards=null message={
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "core_node_name":"core_node2",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"http://127.0.0.1:48846",
   [junit4]   2> 	  "node_name":"127.0.0.1:48846_",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "operation":"state"}
   [junit4]   2> 627023 T4001 C:control_collection S:shard1 c:collection1 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
   [junit4]   2> 627024 T4001 C:control_collection S:shard1 c:collection1 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
   [junit4]   2> 627024 T4001 C:control_collection S:shard1 c:collection1 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
   [junit4]   2> 627025 T4001 C:control_collection S:shard1 c:collection1 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
   [junit4]   2> 627034 T4087 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 627048 T4001 C:control_collection S:shard1 c:collection1 oasc.CachingDirectoryFactory.close Closing StandardDirectoryFactory - 2 directories currently being tracked
   [junit4]   2> 627048 T4001 C:control_collection S:shard1 c:collection1 oasc.CachingDirectoryFactory.closeCacheValue looking to close /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest E55FEF0060F5B0E-001/shard-2-001/cores/collection1/data/index [CachedDir<<refCount=0;path=/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest E55FEF0060F5B0E-001/shard-2-001/cores/collection1/data/index;done=false>>]
   [junit4]   2> 627049 T4001 C:control_collection S:shard1 c:collection1 oasc.CachingDirectoryFactory.close Closing directory: /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest E55FEF0060F5B0E-001/shard-2-001/cores/collection1/data/index
   [junit4]   2> 627049 T4001 C:control_collection S:shard1 c:collection1 oasc.CachingDirectoryFactory.closeCacheValue looking to close /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest E55FEF0060F5B0E-001/shard-2-001/cores/collection1/data [CachedDir<<refCount=0;path=/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest E55FEF0060F5B0E-001/shard-2-001/cores/collection1/data;done=false>>]
   [junit4]   2> 627049 T4001 C:control_collection S:shard1 c:collection1 oasc.CachingDirectoryFactory.close Closing directory: /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest E55FEF0060F5B0E-001/shard-2-001/cores/collection1/data
   [junit4]   2> 627050 T4001 C:control_collection S:shard1 c:collection1 oasc.Overseer.close Overseer (id=93563877340610568-127.0.0.1:48846_-n_0000000002) closing
   [junit4]   2> 627050 T4112 oasc.Overseer$ClusterStateUpdater.run Overseer Loop exiting : 127.0.0.1:48846_
   [junit4]   2> 627053 T4097 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x14c67d9c8b20008, likely client has closed socket
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 
   [junit4]   2> 628553 T4087 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
   [junit4]   2> 628584 T4001 C:control_collection S:shard1 c:collection1 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/,null}
   [junit4]   2> 628775 T4001 C:control_collection S:shard1 c:collection1 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:40183 40183
   [junit4]   2> 628778 T4096 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:40183 40183
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=ChaosMonkeyNothingIsSafeTest -Dtests.method=test -Dtests.seed=E55FEF0060F5B0E -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=ro -Dtests.timezone=Australia/Tasmania -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] FAILURE 38.1s J0 | ChaosMonkeyNothingIsSafeTest.test <<<
   [junit4]    > Throwable #1: java.lang.AssertionError: There were too many update fails (21 > 20) - we expect it can happen, but shouldn't easily
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([E55FEF0060F5B0E:8601C12AA8F336F6]:0)
   [junit4]    > 	at org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest.test(ChaosMonkeyNothingIsSafeTest.java:230)
   [junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:960)
   [junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:935)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 628809 T4001 C:control_collection S:shard1 c:collection1 oas.SolrTestCaseJ4.deleteCore ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest E55FEF0060F5B0E-001
   [junit4]   2> 38120 T4000 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 1 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene50): {rnd_b=FSTOrd50, _version_=FST50, a_t=PostingsFormat(name=LuceneVarGapFixedInterval), a_i=FSTOrd50, id=FSTOrd50}, docValues:{}, sim=DefaultSimilarity, locale=ro, timezone=Australia/Tasmania
   [junit4]   2> NOTE: Linux 3.13.0-46-generic i386/Oracle Corporation 1.8.0_40 (32-bit)/cpus=12,threads=1,free=373020528,total=536870912
   [junit4]   2> NOTE: All tests run in this JVM: [TestSolr4Spatial, SolrXmlInZkTest, AddSchemaFieldsUpdateProcessorFactoryTest, TestDynamicFieldCollectionResource, DistribDocExpirationUpdateProcessorTest, TestRealTimeGet, TestIBSimilarityFactory, TestFunctionQuery, TestLazyCores, TestClusterStateMutator, DateFieldTest, SyncSliceTest, TestNamedUpdateProcessors, SSLMigrationTest, HighlighterConfigTest, MinimalSchemaTest, HdfsBasicDistributedZkTest, OutOfBoxZkACLAndCredentialsProvidersTest, TestFileDictionaryLookup, TestCloudManagedSchemaConcurrent, TestCryptoKeys, ExternalFileFieldSortTest, HdfsUnloadDistributedZkTest, TestInfoStreamLogging, ExitableDirectoryReaderTest, QueryResultKeyTest, TermVectorComponentTest, SimplePostToolTest, TestRandomFaceting, TestEmbeddedSolrServerConstructors, TestCloudPivotFacet, TestIndexingPerformance, FastVectorHighlighterTest, HdfsChaosMonkeySafeLeaderTest, TestSortingResponseWriter, SchemaVersionSpecificBehaviorTest, DistributedFacetPivotLargeTest, SolrRequestParserTest, UpdateRequestProcessorFactoryTest, HighlighterTest, BasicDistributedZkTest, CollectionsAPIAsyncDistributedZkTest, ChaosMonkeyNothingIsSafeTest]
   [junit4] Completed on J0 in 38.90s, 1 test, 1 failure <<< FAILURES!

[...truncated 900 lines...]
BUILD FAILED
/home/jenkins/workspace/Lucene-Solr-5.x-Linux/build.xml:529: The following error occurred while executing this line:
/home/jenkins/workspace/Lucene-Solr-5.x-Linux/build.xml:477: The following error occurred while executing this line:
/home/jenkins/workspace/Lucene-Solr-5.x-Linux/build.xml:61: The following error occurred while executing this line:
/home/jenkins/workspace/Lucene-Solr-5.x-Linux/extra-targets.xml:39: The following error occurred while executing this line:
/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build.xml:191: The following error occurred while executing this line:
/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/common-build.xml:509: The following error occurred while executing this line:
/home/jenkins/workspace/Lucene-Solr-5.x-Linux/lucene/common-build.xml:1370: The following error occurred while executing this line:
/home/jenkins/workspace/Lucene-Solr-5.x-Linux/lucene/common-build.xml:976: There were test failures: 478 suites, 1902 tests, 1 failure, 48 ignored (21 assumptions)

Total time: 53 minutes 55 seconds
Build step 'Invoke Ant' marked build as failure
[description-setter] Description set: Java: 32bit/jdk1.8.0_40 -client -XX:+UseG1GC
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any