You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2014/09/20 11:45:10 UTC

[JENKINS] Lucene-Solr-Tests-5.x-Java7 - Build # 2124 - Failure

Build: https://builds.apache.org/job/Lucene-Solr-Tests-5.x-Java7/2124/

2 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.handler.TestReplicationHandlerBackup

Error Message:
1 thread leaked from SUITE scope at org.apache.solr.handler.TestReplicationHandlerBackup:     1) Thread[id=125, name=Thread-78, state=RUNNABLE, group=TGRP-TestReplicationHandlerBackup]         at java.net.PlainSocketImpl.socketConnect(Native Method)         at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)         at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)         at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)         at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)         at java.net.Socket.connect(Socket.java:579)         at java.net.Socket.connect(Socket.java:528)         at sun.net.NetworkClient.doConnect(NetworkClient.java:180)         at sun.net.www.http.HttpClient.openServer(HttpClient.java:432)         at sun.net.www.http.HttpClient.openServer(HttpClient.java:527)         at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:652)         at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1323)         at java.net.URL.openStream(URL.java:1037)         at org.apache.solr.handler.TestReplicationHandlerBackup$BackupThread.run(TestReplicationHandlerBackup.java:318)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.handler.TestReplicationHandlerBackup: 
   1) Thread[id=125, name=Thread-78, state=RUNNABLE, group=TGRP-TestReplicationHandlerBackup]
        at java.net.PlainSocketImpl.socketConnect(Native Method)
        at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
        at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
        at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
        at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
        at java.net.Socket.connect(Socket.java:579)
        at java.net.Socket.connect(Socket.java:528)
        at sun.net.NetworkClient.doConnect(NetworkClient.java:180)
        at sun.net.www.http.HttpClient.openServer(HttpClient.java:432)
        at sun.net.www.http.HttpClient.openServer(HttpClient.java:527)
        at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:652)
        at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1323)
        at java.net.URL.openStream(URL.java:1037)
        at org.apache.solr.handler.TestReplicationHandlerBackup$BackupThread.run(TestReplicationHandlerBackup.java:318)
	at __randomizedtesting.SeedInfo.seed([5E665365FCEEC0D8]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.handler.TestReplicationHandlerBackup

Error Message:
There are still zombie threads that couldn't be terminated:    1) Thread[id=125, name=Thread-78, state=RUNNABLE, group=TGRP-TestReplicationHandlerBackup]         at java.net.PlainSocketImpl.socketConnect(Native Method)         at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)         at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)         at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)         at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)         at java.net.Socket.connect(Socket.java:579)         at java.net.Socket.connect(Socket.java:528)         at sun.net.NetworkClient.doConnect(NetworkClient.java:180)         at sun.net.www.http.HttpClient.openServer(HttpClient.java:432)         at sun.net.www.http.HttpClient.openServer(HttpClient.java:527)         at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:652)         at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1323)         at java.net.URL.openStream(URL.java:1037)         at org.apache.solr.handler.TestReplicationHandlerBackup$BackupThread.run(TestReplicationHandlerBackup.java:318)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   1) Thread[id=125, name=Thread-78, state=RUNNABLE, group=TGRP-TestReplicationHandlerBackup]
        at java.net.PlainSocketImpl.socketConnect(Native Method)
        at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
        at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
        at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
        at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
        at java.net.Socket.connect(Socket.java:579)
        at java.net.Socket.connect(Socket.java:528)
        at sun.net.NetworkClient.doConnect(NetworkClient.java:180)
        at sun.net.www.http.HttpClient.openServer(HttpClient.java:432)
        at sun.net.www.http.HttpClient.openServer(HttpClient.java:527)
        at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:652)
        at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1323)
        at java.net.URL.openStream(URL.java:1037)
        at org.apache.solr.handler.TestReplicationHandlerBackup$BackupThread.run(TestReplicationHandlerBackup.java:318)
	at __randomizedtesting.SeedInfo.seed([5E665365FCEEC0D8]:0)




Build Log:
[...truncated 10632 lines...]
   [junit4] Suite: org.apache.solr.handler.TestReplicationHandlerBackup
   [junit4]   2> Creating dataDir: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandlerBackup-5E665365FCEEC0D8-001/init-core-data-001
   [junit4]   2> 44004 T103 oas.SolrTestCaseJ4.setUp ###Starting doTestBackup
   [junit4]   2> 44766 T103 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 45066 T103 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:63440
   [junit4]   2> 45208 T103 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 45208 T103 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 45208 T103 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandlerBackup-5E665365FCEEC0D8-001/solr-instance-001
   [junit4]   2> 45209 T103 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandlerBackup-5E665365FCEEC0D8-001/solr-instance-001/'
   [junit4]   2> 45244 T103 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandlerBackup-5E665365FCEEC0D8-001/solr-instance-001/solr.xml
   [junit4]   2> 45313 T103 oasc.CoreContainer.<init> New CoreContainer 1029226339
   [junit4]   2> 45314 T103 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandlerBackup-5E665365FCEEC0D8-001/solr-instance-001/]
   [junit4]   2> 45371 T103 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 45372 T103 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 45372 T103 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 45372 T103 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 45373 T103 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 45373 T103 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 45373 T103 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 45373 T103 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 45374 T103 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 45374 T103 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
   [junit4]   2> 45375 T103 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 45375 T103 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 45375 T103 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 45381 T115 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandlerBackup-5E665365FCEEC0D8-001/solr-instance-001/collection1/'
   [junit4]   2> 45473 T115 oasc.SolrConfig.<init> Using Lucene MatchVersion: 5.0.0
   [junit4]   2> 45513 T115 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 45515 T115 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 45521 T115 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 45538 T115 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 45539 T115 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from instancedir /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandlerBackup-5E665365FCEEC0D8-001/solr-instance-001/collection1/
   [junit4]   2> 45539 T115 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 45540 T115 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandlerBackup-5E665365FCEEC0D8-001/solr-instance-001/collection1/, dataDir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandlerBackup-5E665365FCEEC0D8-001/solr-instance-001/collection1/data/
   [junit4]   2> 45540 T115 oasc.SolrCore.<init> JMX monitoring not detected for core: collection1
   [junit4]   2> 45541 T115 oasc.CachingDirectoryFactory.get return new directory for /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandlerBackup-5E665365FCEEC0D8-001/solr-instance-001/collection1/data
   [junit4]   2> 45541 T115 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandlerBackup-5E665365FCEEC0D8-001/solr-instance-001/collection1/data/index/
   [junit4]   2> 45541 T115 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandlerBackup-5E665365FCEEC0D8-001/solr-instance-001/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 45541 T115 oasc.CachingDirectoryFactory.get return new directory for /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandlerBackup-5E665365FCEEC0D8-001/solr-instance-001/collection1/data/index
   [junit4]   2> 45542 T115 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=40, maxMergeAtOnceExplicit=21, maxMergedSegmentMB=81.8818359375, floorSegmentMB=1.6181640625, forceMergeDeletesPctAllowed=3.3622094348333684, segmentsPerTier=41.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 45542 T115 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory(id=lucene-5e8c1f43)),segFN=segments_1,generation=1}
   [junit4]   2> 45543 T115 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 45544 T115 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 45544 T115 oasc.RequestHandlers.initHandlersFromConfig created /update/csv: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 45544 T115 oasc.RequestHandlers.initHandlersFromConfig created /update/json: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 45545 T115 oasc.RequestHandlers.initHandlersFromConfig created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 45545 T115 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 45613 T115 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 45613 T115 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 45634 T115 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 45636 T115 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 45637 T115 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 45639 T115 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 45641 T115 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 45641 T115 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 45642 T115 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=50, maxMergeAtOnceExplicit=23, maxMergedSegmentMB=96.107421875, floorSegmentMB=0.873046875, forceMergeDeletesPctAllowed=3.790656550787106, segmentsPerTier=28.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.12852688167108958
   [junit4]   2> 45642 T115 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory(id=lucene-5e8c1f43)),segFN=segments_1,generation=1}
   [junit4]   2> 45643 T115 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 45643 T115 oass.SolrIndexSearcher.<init> Opening Searcher@37022ee5[collection1] main
   [junit4]   2> 45644 T115 oasr.ManagedResourceStorage$FileStorageIO.configure File-based storage initialized to use dir: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandlerBackup-5E665365FCEEC0D8-001/solr-instance-001/collection1/conf
   [junit4]   2> 45644 T115 oasr.RestManager.init Initializing RestManager with initArgs: {storageDir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandlerBackup-5E665365FCEEC0D8-001/solr-instance-001/collection1/conf}
   [junit4]   2> 45644 T115 oasr.ManagedResourceStorage.load Reading _rest_managed.json using file:dir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandlerBackup-5E665365FCEEC0D8-001/solr-instance-001/collection1/conf
   [junit4]   2> 45645 T115 oasr.ManagedResource.reloadFromStorage WARN No stored data found for /rest/managed
   [junit4]   2> 45646 T115 oasr.ManagedResourceStorage$JsonStorage.store Saved JSON object to path _rest_managed.json using file:dir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandlerBackup-5E665365FCEEC0D8-001/solr-instance-001/collection1/conf
   [junit4]   2> 45646 T115 oasr.RestManager.init Initializing 0 registered ManagedResources
   [junit4]   2> 45653 T115 oash.ReplicationHandler.inform Replication enabled for following config files: schema-replication2.xml:schema.xml
   [junit4]   2> 45658 T115 oash.ReplicationHandler.inform Commits will be reserved for  10000
   [junit4]   2> 45658 T116 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@37022ee5[collection1] main{UninvertingDirectoryReader()}
   [junit4]   2> 45658 T115 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 45659 T103 oass.SolrDispatchFilter.init user.dir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J1
   [junit4]   2> 45659 T103 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> ASYNC  NEW_CORE C49 name=collection1 org.apache.solr.core.SolrCore@25f07251
   [junit4]   2> 47154 T110 C49 oasu.DirectUpdateHandler2.deleteAll [collection1] REMOVING ALL DOCUMENTS FROM INDEX
   [junit4]   2> 47156 T110 C49 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {deleteByQuery=*:*} 0 46
   [junit4]   2> 47351 T110 C49 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[0]} 0 2
   [junit4]   2> 47356 T110 C49 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[1]} 0 1
   [junit4]   2> 47361 T110 C49 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[2]} 0 1
   [junit4]   2> 47366 T110 C49 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[3]} 0 1
   [junit4]   2> 47371 T110 C49 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[4]} 0 1
   [junit4]   2> 47376 T110 C49 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[5]} 0 1
   [junit4]   2> 47381 T110 C49 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[6]} 0 1
   [junit4]   2> 47387 T110 C49 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[7]} 0 1
   [junit4]   2> 47392 T110 C49 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[8]} 0 0
   [junit4]   2> 47397 T110 C49 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[9]} 0 1
   [junit4]   2> 47401 T110 C49 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[10]} 0 1
   [junit4]   2> 47405 T110 C49 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[11]} 0 1
   [junit4]   2> 47410 T110 C49 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[12]} 0 1
   [junit4]   2> 47414 T110 C49 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[13]} 0 1
   [junit4]   2> 47419 T110 C49 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[14]} 0 0
   [junit4]   2> 47426 T110 C49 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[15]} 0 0
   [junit4]   2> 47430 T110 C49 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[16]} 0 1
   [junit4]   2> 47435 T110 C49 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[17]} 0 1
   [junit4]   2> 47439 T110 C49 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[18]} 0 1
   [junit4]   2> 47443 T110 C49 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[19]} 0 1
   [junit4]   2> 47447 T110 C49 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[20]} 0 1
   [junit4]   2> 47451 T110 C49 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[21]} 0 1
   [junit4]   2> 47455 T110 C49 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[22]} 0 1
   [junit4]   2> 47459 T110 C49 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[23]} 0 1
   [junit4]   2> 47463 T110 C49 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[24]} 0 1
   [junit4]   2> 47467 T110 C49 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[25]} 0 1
   [junit4]   2> 47471 T110 C49 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[26]} 0 1
   [junit4]   2> 47475 T110 C49 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[27]} 0 1
   [junit4]   2> 47480 T110 C49 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[28]} 0 1
   [junit4]   2> 47484 T110 C49 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[29]} 0 1
   [junit4]   2> 47488 T110 C49 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[30]} 0 1
   [junit4]   2> 47492 T110 C49 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[31]} 0 1
   [junit4]   2> 47495 T110 C49 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[32]} 0 0
   [junit4]   2> 47500 T110 C49 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[33]} 0 1
   [junit4]   2> 47504 T110 C49 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[34]} 0 1
   [junit4]   2> 47509 T110 C49 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[35]} 0 1
   [junit4]   2> 47514 T110 C49 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[36]} 0 1
   [junit4]   2> 47519 T110 C49 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[37]} 0 1
   [junit4]   2> 47523 T110 C49 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[38]} 0 1
   [junit4]   2> 47527 T110 C49 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[39]} 0 1
   [junit4]   2> 47531 T110 C49 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[40]} 0 1
   [junit4]   2> 47535 T110 C49 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[41]} 0 1
   [junit4]   2> 47539 T110 C49 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[42]} 0 1
   [junit4]   2> 47543 T110 C49 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[43]} 0 0
   [junit4]   2> 47548 T110 C49 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[44]} 0 1
   [junit4]   2> 47552 T110 C49 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[45]} 0 1
   [junit4]   2> 47556 T110 C49 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[46]} 0 1
   [junit4]   2> 47560 T110 C49 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[47]} 0 0
   [junit4]   2> 47564 T110 C49 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[48]} 0 1
   [junit4]   2> 47568 T110 C49 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[49]} 0 1
   [junit4]   2> 47572 T110 C49 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[50]} 0 1
   [junit4]   2> 47576 T110 C49 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[51]} 0 1
   [junit4]   2> 47580 T110 C49 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[52]} 0 1
   [junit4]   2> 47584 T110 C49 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[53]} 0 1
   [junit4]   2> 47588 T110 C49 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[54]} 0 1
   [junit4]   2> 47592 T110 C49 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[55]} 0 1
   [junit4]   2> 47596 T110 C49 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[56]} 0 1
   [junit4]   2> 47600 T110 C49 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[57]} 0 1
   [junit4]   2> 47604 T110 C49 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[58]} 0 0
   [junit4]   2> 47608 T110 C49 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[59]} 0 1
   [junit4]   2> 47612 T110 C49 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[60]} 0 1
   [junit4]   2> 47616 T110 C49 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[61]} 0 1
   [junit4]   2> 47620 T110 C49 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[62]} 0 1
   [junit4]   2> 47624 T110 C49 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[63]} 0 1
   [junit4]   2> 47628 T110 C49 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[64]} 0 1
   [junit4]   2> 47632 T110 C49 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[65]} 0 1
   [junit4]   2> 47636 T110 C49 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[66]} 0 0
   [junit4]   2> 47641 T110 C49 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[67]} 0 1
   [junit4]   2> 47645 T110 C49 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[68]} 0 0
   [junit4]   2> 47649 T110 C49 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[69]} 0 1
   [junit4]   2> 47653 T110 C49 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[70]} 0 0
   [junit4]   2> 47657 T110 C49 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[71]} 0 1
   [junit4]   2> 47661 T110 C49 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[72]} 0 1
   [junit4]   2> 47665 T110 C49 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[73]} 0 0
   [junit4]   2> 47669 T110 C49 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[74]} 0 1
   [junit4]   2> 47673 T110 C49 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[75]} 0 1
   [junit4]   2> 47677 T110 C49 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[76]} 0 1
   [junit4]   2> 47681 T110 C49 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[77]} 0 1
   [junit4]   2> 47685 T110 C49 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[78]} 0 0
   [junit4]   2> 47720 T110 C49 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 47733 T110 C49 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory(id=lucene-5e8c1f43)),segFN=segments_1,generation=1}
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory(id=lucene-5e8c1f43)),segFN=segments_2,generation=2}
   [junit4]   2> 47734 T110 C49 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
   [junit4]   2> 47761 T110 C49 oass.SolrIndexSearcher.<init> Opening Searcher@69ac0421[collection1] main
   [junit4]   2> 47762 T110 C49 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 47762 T116 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@69ac0421[collection1] main{UninvertingDirectoryReader(Uninverting(_0(5.0.0):C79))}
   [junit4]   2> 47763 T110 C49 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin&waitSearcher=true&commit=true&softCommit=false} {commit=} 0 43
   [junit4]   2> 47909 T111 C49 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=105 
   [junit4]   2> 47929 T108 C49 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={name=jlgroc&command=backup} status=0 QTime=124 
   [junit4]   2> 47929 T120 oash.SnapShooter.createSnapshot Creating backup snapshot...
   [junit4]   2> 47972 T120 oash.SnapShooter.createSnapshot Done creating backup snapshot: jlgroc
   [junit4]   2> 48127 T108 C49 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 49152 T108 C49 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 49154 T111 C49 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={name=bzbdvwfgtme&command=backup} status=0 QTime=2 
   [junit4]   2> 49154 T122 oash.SnapShooter.createSnapshot Creating backup snapshot...
   [junit4]   2> 49167 T122 oash.SnapShooter.createSnapshot Done creating backup snapshot: bzbdvwfgtme
   [junit4]   2> 49356 T111 C49 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 49365 T108 C49 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 49368 T111 C49 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={name=jlgroc&command=deletebackup} status=0 QTime=4 
   [junit4]   2> 49368 T124 oash.SnapShooter.deleteNamedSnapshot Deleting snapshot: jlgroc
   [junit4]   2> 49568 T111 C49 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 49570 T111 C49 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 49571 T103 oas.SolrTestCaseJ4.tearDown ###Ending doTestBackup
   [junit4]   2> 49571 T108 C49 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={name=bzbdvwfgtme&command=deletebackup} status=0 QTime=1 
   [junit4]   2> 49571 T126 oash.SnapShooter.deleteNamedSnapshot Deleting snapshot: bzbdvwfgtme
   [junit4]   2> 49573 T103 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=1029226339
   [junit4]   2> 49574 T103 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/solr,null}
   [junit4]   2> 49581 T108 C49 oasc.SolrException.log ERROR null:org.eclipse.jetty.io.EofException
   [junit4]   2> 		at org.eclipse.jetty.http.HttpGenerator.flushBuffer(HttpGenerator.java:914)
   [junit4]   2> 		at org.eclipse.jetty.http.AbstractGenerator.flush(AbstractGenerator.java:443)
   [junit4]   2> 		at org.eclipse.jetty.server.HttpOutput.flush(HttpOutput.java:100)
   [junit4]   2> 		at org.eclipse.jetty.server.AbstractHttpConnection$Output.flush(AbstractHttpConnection.java:1094)
   [junit4]   2> 		at sun.nio.cs.StreamEncoder.implFlush(StreamEncoder.java:297)
   [junit4]   2> 		at sun.nio.cs.StreamEncoder.flush(StreamEncoder.java:141)
   [junit4]   2> 		at java.io.OutputStreamWriter.flush(OutputStreamWriter.java:229)
   [junit4]   2> 		at org.apache.solr.util.FastWriter.flush(FastWriter.java:137)
   [junit4]   2> 		at org.apache.solr.response.QueryResponseWriterUtil.writeQueryResponse(QueryResponseWriterUtil.java:54)
   [junit4]   2> 		at org.apache.solr.servlet.SolrDispatchFilter.writeResponse(SolrDispatchFilter.java:771)
   [junit4]   2> 		at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:424)
   [junit4]   2> 		at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:202)
   [junit4]   2> 		at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
   [junit4]   2> 		at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:137)
   [junit4]   2> 		at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
   [junit4]   2> 		at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455)
   [junit4]   2> 		at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:229)
   [junit4]   2> 		at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
   [junit4]   2> 		at org.eclipse.jetty.server.handler.GzipHandler.handle(GzipHandler.java:301)
   [junit4]   2> 		at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1077)
   [junit4]   2> 		at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384)
   [junit4]   2> 		at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
   [junit4]   2> 		at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009)
   [junit4]   2> 		at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
   [junit4]   2> 		at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
   [junit4]   2> 		at org.eclipse.jetty.server.Server.handle(Server.java:368)
   [junit4]   2> 		at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489)
   [junit4]   2> 		at org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53)
   [junit4]   2> 		at org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:942)
   [junit4]   2> 		at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:1004)
   [junit4]   2> 		at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:640)
   [junit4]   2> 		at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
   [junit4]   2> 		at org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72)
   [junit4]   2> 		at org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264)
   [junit4]   2> 		at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
   [junit4]   2> 		at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
   [junit4]   2> 		at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 	Caused by: java.net.SocketException: Socket closed
   [junit4]   2> 		at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:121)
   [junit4]   2> 		at java.net.SocketOutputStream.write(SocketOutputStream.java:159)
   [junit4]   2> 		at org.eclipse.jetty.io.ByteArrayBuffer.writeTo(ByteArrayBuffer.java:375)
   [junit4]   2> 		at org.eclipse.jetty.io.bio.StreamEndPoint.flush(StreamEndPoint.java:164)
   [junit4]   2> 		at org.eclipse.jetty.io.bio.StreamEndPoint.flush(StreamEndPoint.java:182)
   [junit4]   2> 		at org.eclipse.jetty.http.HttpGenerator.flushBuffer(HttpGenerator.java:838)
   [junit4]   2> 		... 36 more
   [junit4]   2> 	
   [junit4]   2> 49583 T108 C49 oasc.SolrException.log ERROR null:org.eclipse.jetty.io.EofException
   [junit4]   2> 		at org.eclipse.jetty.http.HttpGenerator.flushBuffer(HttpGenerator.java:914)
   [junit4]   2> 		at org.eclipse.jetty.http.AbstractGenerator.flush(AbstractGenerator.java:443)
   [junit4]   2> 		at org.eclipse.jetty.server.HttpOutput.flush(HttpOutput.java:100)
   [junit4]   2> 		at org.eclipse.jetty.server.AbstractHttpConnection$Output.flush(AbstractHttpConnection.java:1094)
   [junit4]   2> 		at sun.nio.cs.StreamEncoder.implFlush(StreamEncoder.java:297)
   [junit4]   2> 		at sun.nio.cs.StreamEncoder.flush(StreamEncoder.java:141)
   [junit4]   2> 		at java.io.OutputStreamWriter.flush(OutputStreamWriter.java:229)
   [junit4]   2> 		at org.apache.solr.util.FastWriter.flush(FastWriter.java:137)
   [junit4]   2> 		at org.apache.solr.response.QueryResponseWriterUtil.writeQueryResponse(QueryResponseWriterUtil.java:54)
   [junit4]   2> 		at org.apache.solr.servlet.SolrDispatchFilter.writeResponse(SolrDispatchFilter.java:771)
   [junit4]   2> 		at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:424)
   [junit4]   2> 		at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:202)
   [junit4]   2> 		at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
   [junit4]   2> 		at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:137)
   [junit4]   2> 		at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
   [junit4]   2> 		at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455)
   [junit4]   2> 		at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:229)
   [junit4]   2> 		at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
   [junit4]   2> 		at org.eclipse.jetty.server.handler.GzipHandler.handle(GzipHandler.java:301)
   [junit4]   2> 		at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1077)
   [junit4]   2> 		at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384)
   [junit4]   2> 		at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
   [junit4]   2> 		at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009)
   [junit4]   2> 		at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
   [junit4]   2> 		at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
   [junit4]   2> 		at org.eclipse.jetty.server.Server.handle(Server.java:368)
   [junit4]   2> 		at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489)
   [junit4]   2> 		at org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53)
   [junit4]   2> 		at org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:942)
   [junit4]   2> 		at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:1004)
   [junit4]   2> 		at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:640)
   [junit4]   2> 		at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
   [junit4]   2> 		at org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72)
   [junit4]   2> 		at org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264)
   [junit4]   2> 		at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
   [junit4]   2> 		at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
   [junit4]   2> 		at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 	Caused by: java.net.SocketException: Socket closed
   [junit4]   2> 		at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:121)
   [junit4]   2> 		at java.net.SocketOutputStream.write(SocketOutputStream.java:159)
   [junit4]   2> 		at org.eclipse.jetty.io.ByteArrayBuffer.writeTo(ByteArrayBuffer.java:375)
   [junit4]   2> 		at org.eclipse.jetty.io.bio.StreamEndPoint.flush(StreamEndPoint.java:164)
   [junit4]   2> 		at org.eclipse.jetty.io.bio.StreamEndPoint.flush(StreamEndPoint.java:182)
   [junit4]   2> 		at org.eclipse.jetty.http.HttpGenerator.flushBuffer(HttpGenerator.java:838)
   [junit4]   2> 		... 36 more
   [junit4]   2> 	
   [junit4]   2> 49584 T108 C49 oejs.Response.sendError WARN Committed before 500 {msg=Socket closed,trace=org.eclipse.jetty.io.EofException
   [junit4]   2> 		at org.eclipse.jetty.http.HttpGenerator.flushBuffer(HttpGenerator.java:914)
   [junit4]   2> 		at org.eclipse.jetty.http.AbstractGenerator.flush(AbstractGenerator.java:443)
   [junit4]   2> 		at org.eclipse.jetty.server.HttpOutput.flush(HttpOutput.java:100)
   [junit4]   2> 		at org.eclipse.jetty.server.AbstractHttpConnection$Output.flush(AbstractHttpConnection.java:1094)
   [junit4]   2> 		at sun.nio.cs.StreamEncoder.implFlush(StreamEncoder.java:297)
   [junit4]   2> 		at sun.nio.cs.StreamEncoder.flush(StreamEncoder.java:141)
   [junit4]   2> 		at java.io.OutputStreamWriter.flush(OutputStreamWriter.java:229)
   [junit4]   2> 		at org.apache.solr.util.FastWriter.flush(FastWriter.java:137)
   [junit4]   2> 		at org.apache.solr.response.QueryResponseWriterUtil.writeQueryResponse(QueryResponseWriterUtil.java:54)
   [junit4]   2> 		at org.apache.solr.servlet.SolrDispatchFilter.writeResponse(SolrDispatchFilter.java:771)
   [junit4]   2> 		at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:424)
   [junit4]   2> 		at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:202)
   [junit4]   2> 		at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
   [junit4]   2> 		at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:137)
   [junit4]   2> 		at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
   [junit4]   2> 		at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455)
   [junit4]   2> 		at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:229)
   [junit4]   2> 		at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
   [junit4]   2> 		at org.eclipse.jetty.server.handler.GzipHandler.handle(GzipHandler.java:301)
   [junit4]   2> 		at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1077)
   [junit4]   2> 		at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384)
   [junit4]   2> 		at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
   [junit4]   2> 		at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009)
   [junit4]   2> 		at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
   [junit4]   2> 		at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
   [junit4]   2> 		at org.eclipse.jetty.server.Server.handle(Server.java:368)
   [junit4]   2> 		at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489)
   [junit4]   2> 		at org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53)
   [junit4]   2> 		at org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:942)
   [junit4]   2> 		at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:1004)
   [junit4]   2> 		at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:640)
   [junit4]   2> 		at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
   [junit4]   2> 		at org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72)
   [junit4]   2> 		at org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264)
   [junit4]   2> 		at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
   [junit4]   2> 		at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
   [junit4]   2> 		at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 	Caused by: java.net.SocketException: Socket closed
   [junit4]   2> 		at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:121)
   [junit4]   2> 		at java.net.SocketOutputStream.write(SocketOutputStream.java:159)
   [junit4]   2> 		at org.eclipse.jetty.io.ByteArrayBuffer.writeTo(ByteArrayBuffer.java:375)
   [junit4]   2> 		at org.eclipse.jetty.io.bio.StreamEndPoint.flush(StreamEndPoint.java:164)
   [junit4]   2> 		at org.eclipse.jetty.io.bio.StreamEndPoint.flush(StreamEndPoint.java:182)
   [junit4]   2> 		at org.eclipse.jetty.http.HttpGenerator.flushBuffer(HttpGenerator.java:838)
   [junit4]   2> 		... 36 more
   [junit4]   2> 	,code=500}
   [junit4]   2> 49586 T108 C49 oasc.SolrCore.close [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@25f07251
   [junit4]   2> 49586 T108 C49 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=1,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=79,cumulative_deletesById=0,cumulative_deletesByQuery=1,cumulative_errors=0}
   [junit4]   2> 49587 T108 C49 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
   [junit4]   2> 49587 T108 C49 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
   [junit4]   2> 49588 T108 C49 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
   [junit4]   2> 49589 T108 C49 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
   [junit4]   2> 49589 T108 C49 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
   [junit4]   2> 49590 T108 C49 oasc.CachingDirectoryFactory.closeCacheValue looking to close /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandlerBackup-5E665365FCEEC0D8-001/solr-instance-001/collection1/data/index [CachedDir<<refCount=0;path=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandlerBackup-5E665365FCEEC0D8-001/solr-instance-001/collection1/data/index;done=false>>]
   [junit4]   2> 49590 T108 C49 oasc.CachingDirectoryFactory.close Closing directory: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandlerBackup-5E665365FCEEC0D8-001/solr-instance-001/collection1/data/index
   [junit4]   2> 49590 T108 C49 oasc.CachingDirectoryFactory.closeCacheValue looking to close /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandlerBackup-5E665365FCEEC0D8-001/solr-instance-001/collection1/data [CachedDir<<refCount=0;path=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandlerBackup-5E665365FCEEC0D8-001/solr-instance-001/collection1/data;done=false>>]
   [junit4]   2> 49591 T108 C49 oasc.CachingDirectoryFactory.close Closing directory: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandlerBackup-5E665365FCEEC0D8-001/solr-instance-001/collection1/data
   [junit4]   2> 49592 T108 oejs.ServletHandler.doHandle WARN /solr/replication java.lang.IllegalStateException: Committed
   [junit4]   2> 	at org.eclipse.jetty.server.Response.resetBuffer(Response.java:1144)
   [junit4]   2> 	at org.eclipse.jetty.server.Response.sendError(Response.java:314)
   [junit4]   2> 	at org.apache.solr.servlet.SolrDispatchFilter.sendError(SolrDispatchFilter.java:825)
   [junit4]   2> 	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:432)
   [junit4]   2> 	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:202)
   [junit4]   2> 	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
   [junit4]   2> 	at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:137)
   [junit4]   2> 	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
   [junit4]   2> 	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455)
   [junit4]   2> 	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:229)
   [junit4]   2> 	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
   [junit4]   2> 	at org.eclipse.jetty.server.handler.GzipHandler.handle(GzipHandler.java:301)
   [junit4]   2> 	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1077)
   [junit4]   2> 	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384)
   [junit4]   2> 	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
   [junit4]   2> 	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009)
   [junit4]   2> 	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
   [junit4]   2> 	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
   [junit4]   2> 	at org.eclipse.jetty.server.Server.handle(Server.java:368)
   [junit4]   2> 	at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489)
   [junit4]   2> 	at org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53)
   [junit4]   2> 	at org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:942)
   [junit4]   2> 	at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:1004)
   [junit4]   2> 	at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:640)
   [junit4]   2> 	at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
   [junit4]   2> 	at org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72)
   [junit4]   2> 	at org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264)
   [junit4]   2> 	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
   [junit4]   2> 	at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 
   [junit4]   2> 54599 T103 oas.SolrTestCaseJ4.deleteCore ###deleteCore
   [junit4]   2> 10610 T102 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 1 leaked thread(s).
   [junit4]   2> 30693 T102 ccr.ThreadLeakControl.checkThreadLeaks SEVERE 1 thread leaked from SUITE scope at org.apache.solr.handler.TestReplicationHandlerBackup: 
   [junit4]   2> 	   1) Thread[id=125, name=Thread-78, state=RUNNABLE, group=TGRP-TestReplicationHandlerBackup]
   [junit4]   2> 	        at java.net.PlainSocketImpl.socketConnect(Native Method)
   [junit4]   2> 	        at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
   [junit4]   2> 	        at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
   [junit4]   2> 	        at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
   [junit4]   2> 	        at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
   [junit4]   2> 	        at java.net.Socket.connect(Socket.java:579)
   [junit4]   2> 	        at java.net.Socket.connect(Socket.java:528)
   [junit4]   2> 	        at sun.net.NetworkClient.doConnect(NetworkClient.java:180)
   [junit4]   2> 	        at sun.net.www.http.HttpClient.openServer(HttpClient.java:432)
   [junit4]   2> 	        at sun.net.www.http.HttpClient.openServer(HttpClient.java:527)
   [junit4]   2> 	        at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:652)
   [junit4]   2> 	        at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1323)
   [junit4]   2> 	        at java.net.URL.openStream(URL.java:1037)
   [junit4]   2> 	        at org.apache.solr.handler.TestReplicationHandlerBackup$BackupThread.run(TestReplicationHandlerBackup.java:318)
   [junit4]   2> 30694 T102 ccr.ThreadLeakControl.tryToInterruptAll Starting to interrupt leaked threads:
   [junit4]   2> 	   1) Thread[id=125, name=Thread-78, state=RUNNABLE, group=TGRP-TestReplicationHandlerBackup]
   [junit4]   2> 33701 T102 ccr.ThreadLeakControl.tryToInterruptAll SEVERE There are still zombie threads that couldn't be terminated:
   [junit4]   2> 	   1) Thread[id=125, name=Thread-78, state=RUNNABLE, group=TGRP-TestReplicationHandlerBackup]
   [junit4]   2> 	        at java.net.PlainSocketImpl.socketConnect(Native Method)
   [junit4]   2> 	        at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
   [junit4]   2> 	        at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
   [junit4]   2> 	        at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
   [junit4]   2> 	        at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
   [junit4]   2> 	        at java.net.Socket.connect(Socket.java:579)
   [junit4]   2> 	        at java.net.Socket.connect(Socket.java:528)
   [junit4]   2> 	        at sun.net.NetworkClient.doConnect(NetworkClient.java:180)
   [junit4]   2> 	        at sun.net.www.http.HttpClient.openServer(HttpClient.java:432)
   [junit4]   2> 	        at sun.net.www.http.HttpClient.openServer(HttpClient.java:527)
   [junit4]   2> 	        at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:652)
   [junit4]   2> 	        at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1323)
   [junit4]   2> 	        at java.net.URL.openStream(URL.java:1037)
   [junit4]   2> 	        at org.apache.solr.handler.TestReplicationHandlerBackup$BackupThread.run(TestReplicationHandlerBackup.java:318)
   [junit4]   2> NOTE: test params are: codec=Lucene410: {id=PostingsFormat(name=SimpleText), name=PostingsFormat(name=SimpleText)}, docValues:{}, sim=DefaultSimilarity, locale=es_EC, timezone=America/Cuiaba
   [junit4]   2> NOTE: FreeBSD 9.1-RELEASE-p3 amd64/Oracle Corporation 1.7.0_65 (64-bit)/cpus=16,threads=2,free=172745392,total=297271296
   [junit4]   2> NOTE: All tests run in this JVM: [TestStressVersions, TestPostingsSolrHighlighter, TestIndexingPerformance, TestReplicationHandlerBackup]
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestReplicationHandlerBackup -Dtests.seed=5E665365FCEEC0D8 -Dtests.multiplier=2 -Dtests.slow=true -Dtests.locale=es_EC -Dtests.timezone=America/Cuiaba -Dtests.file.encoding=US-ASCII
   [junit4] ERROR   0.00s J1 | TestReplicationHandlerBackup (suite) <<<
   [junit4]    > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.handler.TestReplicationHandlerBackup: 
   [junit4]    >    1) Thread[id=125, name=Thread-78, state=RUNNABLE, group=TGRP-TestReplicationHandlerBackup]
   [junit4]    >         at java.net.PlainSocketImpl.socketConnect(Native Method)
   [junit4]    >         at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
   [junit4]    >         at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
   [junit4]    >         at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
   [junit4]    >         at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
   [junit4]    >         at java.net.Socket.connect(Socket.java:579)
   [junit4]    >         at java.net.Socket.connect(Socket.java:528)
   [junit4]    >         at sun.net.NetworkClient.doConnect(NetworkClient.java:180)
   [junit4]    >         at sun.net.www.http.HttpClient.openServer(HttpClient.java:432)
   [junit4]    >         at sun.net.www.http.HttpClient.openServer(HttpClient.java:527)
   [junit4]    >         at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:652)
   [junit4]    >         at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1323)
   [junit4]    >         at java.net.URL.openStream(URL.java:1037)
   [junit4]    >         at org.apache.solr.handler.TestReplicationHandlerBackup$BackupThread.run(TestReplicationHandlerBackup.java:318)
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([5E665365FCEEC0D8]:0)Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   [junit4]    >    1) Thread[id=125, name=Thread-78, state=RUNNABLE, group=TGRP-TestReplicationHandlerBackup]
   [junit4]    >         at java.net.PlainSocketImpl.socketConnect(Native Method)
   [junit4]    >         at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
   [junit4]    >         at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
   [junit4]    >         at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
   [junit4]    >         at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
   [junit4]    >         at java.net.Socket.connect(Socket.java:579)
   [junit4]    >         at java.net.Socket.connect(Socket.java:528)
   [junit4]    >         at sun.net.NetworkClient.doConnect(NetworkClient.java:180)
   [junit4]    >         at sun.net.www.http.HttpClient.openServer(HttpClient.java:432)
   [junit4]    >         at sun.net.www.http.HttpClient.openServer(HttpClient.java:527)
   [junit4]    >         at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:652)
   [junit4]    >         at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1323)
   [junit4]    >         at java.net.URL.openStream(URL.java:1037)
   [junit4]    >         at org.apache.solr.handler.TestReplicationHandlerBackup$BackupThread.run(TestReplicationHandlerBackup.java:318)
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([5E665365FCEEC0D8]:0)
   [junit4] Completed on J1 in 33.84s, 1 test, 2 errors <<< FAILURES!

[...truncated 1296 lines...]
BUILD FAILED
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/build.xml:491: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/build.xml:471: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/build.xml:60: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/extra-targets.xml:39: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build.xml:189: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/common-build.xml:496: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/lucene/common-build.xml:1359: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/lucene/common-build.xml:962: There were test failures: 429 suites, 1772 tests, 2 suite-level errors, 1054 ignored (22 assumptions)

Total time: 41 minutes 7 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Sending artifact delta relative to Lucene-Solr-Tests-5.x-Java7 #2123
Archived 5 artifacts
Archive block size is 32768
Received 0 blocks and 73701747 bytes
Compression is 0.0%
Took 12 sec
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure