You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2014/08/20 20:13:03 UTC

[JENKINS] Lucene-Solr-Tests-trunk-Java7 - Build # 4807 - Failure

Build: https://builds.apache.org/job/Lucene-Solr-Tests-trunk-Java7/4807/

All tests passed

Build Log:
[...truncated 46637 lines...]
-documentation-lint:
     [echo] checking for broken html...
    [jtidy] Checking for broken html (such as invalid tags)...
   [delete] Deleting directory /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/lucene/build/jtidy_tmp
     [echo] Checking for broken links...
     [exec] 
     [exec] Crawl/parse...
     [exec] 
     [exec] Verify...
     [echo] Checking for missing docs...
     [exec] 
     [exec] build/docs/classification/org/apache/lucene/classification/SimpleNaiveBayesClassifier.html
     [exec]   missing Fields: analyzer
     [exec]   missing Fields: atomicReader
     [exec]   missing Fields: classFieldName
     [exec]   missing Fields: indexSearcher
     [exec]   missing Fields: query
     [exec]   missing Fields: textFieldNames
     [exec]   missing Methods: countDocsWithClass()
     [exec]   missing Methods: tokenizeDoc(java.lang.String)
     [exec] 
     [exec] Missing javadocs were found!

BUILD FAILED
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:474: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:63: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/lucene/build.xml:212: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/lucene/build.xml:242: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/lucene/common-build.xml:2425: exec returned: 1

Total time: 155 minutes 30 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Sending artifact delta relative to Lucene-Solr-Tests-trunk-Java7 #4806
Archived 1 artifacts
Archive block size is 32768
Received 0 blocks and 464 bytes
Compression is 0.0%
Took 25 ms
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-Tests-trunk-Java7 - Build # 4808 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-trunk-Java7/4808/

2 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.handler.TestReplicationHandlerBackup

Error Message:
1 thread leaked from SUITE scope at org.apache.solr.handler.TestReplicationHandlerBackup:     1) Thread[id=18419, name=Thread-2467, state=RUNNABLE, group=TGRP-TestReplicationHandlerBackup]         at java.net.PlainSocketImpl.socketConnect(Native Method)         at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)         at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)         at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)         at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)         at java.net.Socket.connect(Socket.java:579)         at java.net.Socket.connect(Socket.java:528)         at sun.net.NetworkClient.doConnect(NetworkClient.java:180)         at sun.net.www.http.HttpClient.openServer(HttpClient.java:432)         at sun.net.www.http.HttpClient.openServer(HttpClient.java:527)         at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:652)         at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1323)         at java.net.URL.openStream(URL.java:1037)         at org.apache.solr.handler.TestReplicationHandlerBackup$BackupThread.run(TestReplicationHandlerBackup.java:313)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.handler.TestReplicationHandlerBackup: 
   1) Thread[id=18419, name=Thread-2467, state=RUNNABLE, group=TGRP-TestReplicationHandlerBackup]
        at java.net.PlainSocketImpl.socketConnect(Native Method)
        at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
        at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
        at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
        at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
        at java.net.Socket.connect(Socket.java:579)
        at java.net.Socket.connect(Socket.java:528)
        at sun.net.NetworkClient.doConnect(NetworkClient.java:180)
        at sun.net.www.http.HttpClient.openServer(HttpClient.java:432)
        at sun.net.www.http.HttpClient.openServer(HttpClient.java:527)
        at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:652)
        at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1323)
        at java.net.URL.openStream(URL.java:1037)
        at org.apache.solr.handler.TestReplicationHandlerBackup$BackupThread.run(TestReplicationHandlerBackup.java:313)
	at __randomizedtesting.SeedInfo.seed([4B12C10A5FB99CCE]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.handler.TestReplicationHandlerBackup

Error Message:
There are still zombie threads that couldn't be terminated:    1) Thread[id=18419, name=Thread-2467, state=RUNNABLE, group=TGRP-TestReplicationHandlerBackup]         at java.net.PlainSocketImpl.socketConnect(Native Method)         at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)         at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)         at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)         at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)         at java.net.Socket.connect(Socket.java:579)         at java.net.Socket.connect(Socket.java:528)         at sun.net.NetworkClient.doConnect(NetworkClient.java:180)         at sun.net.www.http.HttpClient.openServer(HttpClient.java:432)         at sun.net.www.http.HttpClient.openServer(HttpClient.java:527)         at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:652)         at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1323)         at java.net.URL.openStream(URL.java:1037)         at org.apache.solr.handler.TestReplicationHandlerBackup$BackupThread.run(TestReplicationHandlerBackup.java:313)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   1) Thread[id=18419, name=Thread-2467, state=RUNNABLE, group=TGRP-TestReplicationHandlerBackup]
        at java.net.PlainSocketImpl.socketConnect(Native Method)
        at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
        at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
        at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
        at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
        at java.net.Socket.connect(Socket.java:579)
        at java.net.Socket.connect(Socket.java:528)
        at sun.net.NetworkClient.doConnect(NetworkClient.java:180)
        at sun.net.www.http.HttpClient.openServer(HttpClient.java:432)
        at sun.net.www.http.HttpClient.openServer(HttpClient.java:527)
        at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:652)
        at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1323)
        at java.net.URL.openStream(URL.java:1037)
        at org.apache.solr.handler.TestReplicationHandlerBackup$BackupThread.run(TestReplicationHandlerBackup.java:313)
	at __randomizedtesting.SeedInfo.seed([4B12C10A5FB99CCE]:0)




Build Log:
[...truncated 11584 lines...]
   [junit4] Suite: org.apache.solr.handler.TestReplicationHandlerBackup
   [junit4]   2> Creating dataDir: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.handler.TestReplicationHandlerBackup-4B12C10A5FB99CCE-001/init-core-data-001
   [junit4]   2> 1345163 T18399 oas.SolrTestCaseJ4.setUp ###Starting doTestBackup
   [junit4]   2> 1345186 T18399 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 1345190 T18399 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:22814
   [junit4]   2> 1345190 T18399 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 1345191 T18399 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1345191 T18399 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.handler.TestReplicationHandlerBackup-4B12C10A5FB99CCE-001/solr-instance-001
   [junit4]   2> 1345192 T18399 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.handler.TestReplicationHandlerBackup-4B12C10A5FB99CCE-001/solr-instance-001/'
   [junit4]   2> 1345233 T18399 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.handler.TestReplicationHandlerBackup-4B12C10A5FB99CCE-001/solr-instance-001/solr.xml
   [junit4]   2> 1345308 T18399 oasc.CoreContainer.<init> New CoreContainer 2073067869
   [junit4]   2> 1345308 T18399 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.handler.TestReplicationHandlerBackup-4B12C10A5FB99CCE-001/solr-instance-001/]
   [junit4]   2> 1345310 T18399 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 1345310 T18399 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 1345310 T18399 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 1345311 T18399 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 1345311 T18399 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 1345311 T18399 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 1345312 T18399 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 1345312 T18399 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 1345312 T18399 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 1345313 T18399 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
   [junit4]   2> 1345313 T18399 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1345314 T18399 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1345314 T18399 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 1345319 T18409 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.handler.TestReplicationHandlerBackup-4B12C10A5FB99CCE-001/solr-instance-001/collection1/'
   [junit4]   2> 1345367 T18409 oasc.SolrConfig.<init> Using Lucene MatchVersion: 5.0.0
   [junit4]   2> 1345409 T18409 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1345410 T18409 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 1345416 T18409 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 1345434 T18409 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 1345435 T18409 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from instancedir /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.handler.TestReplicationHandlerBackup-4B12C10A5FB99CCE-001/solr-instance-001/collection1/
   [junit4]   2> 1345435 T18409 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 1345435 T18409 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.handler.TestReplicationHandlerBackup-4B12C10A5FB99CCE-001/solr-instance-001/collection1/, dataDir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.handler.TestReplicationHandlerBackup-4B12C10A5FB99CCE-001/solr-instance-001/collection1/data/
   [junit4]   2> 1345436 T18409 oasc.SolrCore.<init> JMX monitoring not detected for core: collection1
   [junit4]   2> 1345436 T18409 oasc.CachingDirectoryFactory.get return new directory for /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.handler.TestReplicationHandlerBackup-4B12C10A5FB99CCE-001/solr-instance-001/collection1/data
   [junit4]   2> 1345437 T18409 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.handler.TestReplicationHandlerBackup-4B12C10A5FB99CCE-001/solr-instance-001/collection1/data/index/
   [junit4]   2> 1345437 T18409 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.handler.TestReplicationHandlerBackup-4B12C10A5FB99CCE-001/solr-instance-001/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 1345437 T18409 oasc.CachingDirectoryFactory.get return new directory for /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.handler.TestReplicationHandlerBackup-4B12C10A5FB99CCE-001/solr-instance-001/collection1/data/index
   [junit4]   2> 1345438 T18409 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=44, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.28606070033303965]
   [junit4]   2> 1345439 T18409 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@59207b90 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@76d5be87),segFN=segments_1,generation=1}
   [junit4]   2> 1345439 T18409 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 1345440 T18409 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 1345441 T18409 oasc.RequestHandlers.initHandlersFromConfig created /update: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 1345441 T18409 oasc.RequestHandlers.initHandlersFromConfig created /update/json: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 1345441 T18409 oasc.RequestHandlers.initHandlersFromConfig created /update/csv: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 1345442 T18409 oasc.RequestHandlers.initHandlersFromConfig created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 1345442 T18409 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 1345443 T18409 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 1345443 T18409 oasc.RequestHandlers.initHandlersFromConfig WARN Multiple requestHandler registered to the same name: /update ignoring: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 1345443 T18409 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 1345461 T18409 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 1345463 T18409 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 1345465 T18409 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 1345467 T18409 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 1345469 T18409 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 1345472 T18409 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 1345472 T18409 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 1345472 T18409 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=1099649749, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
   [junit4]   2> 1345473 T18409 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@59207b90 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@76d5be87),segFN=segments_1,generation=1}
   [junit4]   2> 1345473 T18409 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 1345474 T18409 oass.SolrIndexSearcher.<init> Opening Searcher@187c26d7[collection1] main
   [junit4]   2> 1345474 T18409 oasr.ManagedResourceStorage$FileStorageIO.configure File-based storage initialized to use dir: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.handler.TestReplicationHandlerBackup-4B12C10A5FB99CCE-001/solr-instance-001/collection1/conf
   [junit4]   2> 1345475 T18409 oasr.RestManager.init Initializing RestManager with initArgs: {storageDir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.handler.TestReplicationHandlerBackup-4B12C10A5FB99CCE-001/solr-instance-001/collection1/conf}
   [junit4]   2> 1345475 T18409 oasr.ManagedResourceStorage.load Reading _rest_managed.json using file:dir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.handler.TestReplicationHandlerBackup-4B12C10A5FB99CCE-001/solr-instance-001/collection1/conf
   [junit4]   2> 1345476 T18409 oasr.ManagedResource.reloadFromStorage WARN No stored data found for /rest/managed
   [junit4]   2> 1345476 T18409 oasr.ManagedResourceStorage$JsonStorage.store Saved JSON object to path _rest_managed.json using file:dir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.handler.TestReplicationHandlerBackup-4B12C10A5FB99CCE-001/solr-instance-001/collection1/conf
   [junit4]   2> 1345477 T18409 oasr.RestManager.init Initializing 0 registered ManagedResources
   [junit4]   2> 1345477 T18409 oash.ReplicationHandler.inform Replication enabled for following config files: schema-replication2.xml:schema.xml
   [junit4]   2> 1345477 T18409 oash.ReplicationHandler.inform Commits will be reserved for  10000
   [junit4]   2> 1345478 T18409 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 1345478 T18410 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@187c26d7[collection1] main{UninvertingDirectoryReader()}
   [junit4]   2> 1345478 T18399 oass.SolrDispatchFilter.init user.dir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1
   [junit4]   2> 1345479 T18399 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> ASYNC  NEW_CORE C3260 name=collection1 org.apache.solr.core.SolrCore@52f113a
   [junit4]   2> 1345481 T18402 C3260 oasu.DirectUpdateHandler2.deleteAll [collection1] REMOVING ALL DOCUMENTS FROM INDEX
   [junit4]   2> 1345482 T18402 C3260 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {deleteByQuery=*:*} 0 1
   [junit4]   2> 1345485 T18402 C3260 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[0]} 0 0
   [junit4]   2> 1345488 T18402 C3260 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[1]} 0 0
   [junit4]   2> 1345491 T18402 C3260 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[2]} 0 0
   [junit4]   2> 1345493 T18402 C3260 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[3]} 0 0
   [junit4]   2> 1345495 T18402 C3260 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[4]} 0 0
   [junit4]   2> 1345497 T18402 C3260 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[5]} 0 0
   [junit4]   2> 1345500 T18402 C3260 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[6]} 0 0
   [junit4]   2> 1345503 T18402 C3260 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[7]} 0 0
   [junit4]   2> 1345505 T18402 C3260 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[8]} 0 0
   [junit4]   2> 1345508 T18402 C3260 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[9]} 0 0
   [junit4]   2> 1345511 T18402 C3260 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[10]} 0 0
   [junit4]   2> 1345513 T18402 C3260 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[11]} 0 0
   [junit4]   2> 1345515 T18402 C3260 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[12]} 0 0
   [junit4]   2> 1345518 T18402 C3260 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[13]} 0 0
   [junit4]   2> 1345521 T18402 C3260 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[14]} 0 0
   [junit4]   2> 1345523 T18402 C3260 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[15]} 0 0
   [junit4]   2> 1345526 T18402 C3260 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[16]} 0 0
   [junit4]   2> 1345528 T18402 C3260 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[17]} 0 0
   [junit4]   2> 1345530 T18402 C3260 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[18]} 0 0
   [junit4]   2> 1345533 T18402 C3260 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[19]} 0 0
   [junit4]   2> 1345535 T18402 C3260 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[20]} 0 0
   [junit4]   2> 1345537 T18402 C3260 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[21]} 0 0
   [junit4]   2> 1345539 T18402 C3260 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[22]} 0 0
   [junit4]   2> 1345541 T18402 C3260 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[23]} 0 0
   [junit4]   2> 1345544 T18402 C3260 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[24]} 0 0
   [junit4]   2> 1345546 T18402 C3260 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[25]} 0 0
   [junit4]   2> 1345548 T18402 C3260 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[26]} 0 0
   [junit4]   2> 1345550 T18402 C3260 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[27]} 0 0
   [junit4]   2> 1345552 T18402 C3260 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[28]} 0 0
   [junit4]   2> 1345554 T18402 C3260 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[29]} 0 0
   [junit4]   2> 1345556 T18402 C3260 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[30]} 0 0
   [junit4]   2> 1345559 T18402 C3260 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[31]} 0 0
   [junit4]   2> 1345561 T18402 C3260 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[32]} 0 0
   [junit4]   2> 1345563 T18402 C3260 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[33]} 0 0
   [junit4]   2> 1345565 T18402 C3260 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[34]} 0 0
   [junit4]   2> 1345567 T18402 C3260 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[35]} 0 0
   [junit4]   2> 1345569 T18402 C3260 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[36]} 0 0
   [junit4]   2> 1345571 T18402 C3260 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[37]} 0 0
   [junit4]   2> 1345573 T18402 C3260 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[38]} 0 0
   [junit4]   2> 1345575 T18402 C3260 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[39]} 0 0
   [junit4]   2> 1345578 T18402 C3260 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[40]} 0 0
   [junit4]   2> 1345580 T18402 C3260 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[41]} 0 0
   [junit4]   2> 1345582 T18402 C3260 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[42]} 0 0
   [junit4]   2> 1345584 T18402 C3260 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[43]} 0 0
   [junit4]   2> 1345586 T18402 C3260 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[44]} 0 0
   [junit4]   2> 1345588 T18402 C3260 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[45]} 0 0
   [junit4]   2> 1345590 T18402 C3260 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[46]} 0 0
   [junit4]   2> 1345593 T18402 C3260 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[47]} 0 0
   [junit4]   2> 1345595 T18402 C3260 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[48]} 0 0
   [junit4]   2> 1345597 T18402 C3260 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[49]} 0 0
   [junit4]   2> 1345599 T18402 C3260 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[50]} 0 0
   [junit4]   2> 1345601 T18402 C3260 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[51]} 0 0
   [junit4]   2> 1345604 T18402 C3260 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[52]} 0 0
   [junit4]   2> 1345607 T18402 C3260 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[53]} 0 0
   [junit4]   2> 1345610 T18402 C3260 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[54]} 0 0
   [junit4]   2> 1345613 T18402 C3260 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[55]} 0 0
   [junit4]   2> 1345616 T18402 C3260 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[56]} 0 0
   [junit4]   2> 1345619 T18402 C3260 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[57]} 0 0
   [junit4]   2> 1345621 T18402 C3260 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[58]} 0 0
   [junit4]   2> 1345624 T18402 C3260 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[59]} 0 0
   [junit4]   2> 1345626 T18402 C3260 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[60]} 0 0
   [junit4]   2> 1345629 T18402 C3260 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[61]} 0 0
   [junit4]   2> 1345632 T18402 C3260 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[62]} 0 0
   [junit4]   2> 1345635 T18402 C3260 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[63]} 0 0
   [junit4]   2> 1345638 T18402 C3260 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 1345643 T18402 C3260 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@59207b90 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@76d5be87),segFN=segments_1,generation=1}
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@59207b90 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@76d5be87),segFN=segments_2,generation=2}
   [junit4]   2> 1345644 T18402 C3260 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
   [junit4]   2> 1345646 T18402 C3260 oass.SolrIndexSearcher.<init> Opening Searcher@77e46947[collection1] main
   [junit4]   2> 1345646 T18402 C3260 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 1345646 T18410 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@77e46947[collection1] main{UninvertingDirectoryReader(Uninverting(_0(5.0.0):c64))}
   [junit4]   2> 1345647 T18402 C3260 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={waitSearcher=true&softCommit=false&version=2&wt=javabin&commit=true} {commit=} 0 9
   [junit4]   2> 1345653 T18404 C3260 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 1345661 T18403 C3260 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=backup&name=kfb} status=0 QTime=10 
   [junit4]   2> 1345661 T18414 oash.SnapShooter.createSnapshot Creating backup snapshot...
   [junit4]   2> 1345672 T18414 oash.SnapShooter.createSnapshot Done creating backup snapshot: kfb
   [junit4]   2> 1345859 T18403 C3260 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 1346870 T18403 C3260 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 1346872 T18404 C3260 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=backup&name=kkgvwdbn} status=0 QTime=2 
   [junit4]   2> 1346872 T18416 oash.SnapShooter.createSnapshot Creating backup snapshot...
   [junit4]   2> 1346880 T18416 oash.SnapShooter.createSnapshot Done creating backup snapshot: kkgvwdbn
   [junit4]   2> 1347072 T18404 C3260 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 1347079 T18404 C3260 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=deletebackup&name=kfb} status=0 QTime=1 
   [junit4]   2> 1347079 T18418 oash.SnapShooter.deleteNamedSnapshot Deleting snapshot: kfb
   [junit4]   2> 1347080 T18403 C3260 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 1347283 T18403 C3260 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 1347285 T18403 C3260 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 1347286 T18399 oas.SolrTestCaseJ4.tearDown ###Ending doTestBackup
   [junit4]   2> 1347286 T18420 oash.SnapShooter.deleteNamedSnapshot Deleting snapshot: kkgvwdbn
   [junit4]   2> 1347286 T18404 C3260 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=deletebackup&name=kkgvwdbn} status=0 QTime=1 
   [junit4]   2> 1347287 T18399 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=2073067869
   [junit4]   2> 1347289 T18399 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/solr,null}
   [junit4]   2> 1347290 T18404 C3260 oasc.SolrException.log ERROR null:org.eclipse.jetty.io.EofException
   [junit4]   2> 		at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:142)
   [junit4]   2> 		at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:107)
   [junit4]   2> 		at sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:221)
   [junit4]   2> 		at sun.nio.cs.StreamEncoder.implFlushBuffer(StreamEncoder.java:291)
   [junit4]   2> 		at sun.nio.cs.StreamEncoder.implFlush(StreamEncoder.java:295)
   [junit4]   2> 		at sun.nio.cs.StreamEncoder.flush(StreamEncoder.java:141)
   [junit4]   2> 		at java.io.OutputStreamWriter.flush(OutputStreamWriter.java:229)
   [junit4]   2> 		at org.apache.solr.util.FastWriter.flush(FastWriter.java:137)
   [junit4]   2> 		at org.apache.solr.servlet.SolrDispatchFilter.writeResponse(SolrDispatchFilter.java:769)
   [junit4]   2> 		at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:428)
   [junit4]   2> 		at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:208)
   [junit4]   2> 		at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
   [junit4]   2> 		at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:137)
   [junit4]   2> 		at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
   [junit4]   2> 		at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455)
   [junit4]   2> 		at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:229)
   [junit4]   2> 		at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
   [junit4]   2> 		at org.eclipse.jetty.server.handler.GzipHandler.handle(GzipHandler.java:301)
   [junit4]   2> 		at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1077)
   [junit4]   2> 		at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384)
   [junit4]   2> 		at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
   [junit4]   2> 		at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009)
   [junit4]   2> 		at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
   [junit4]   2> 		at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
   [junit4]   2> 		at org.eclipse.jetty.server.Server.handle(Server.java:368)
   [junit4]   2> 		at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489)
   [junit4]   2> 		at org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53)
   [junit4]   2> 		at org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:942)
   [junit4]   2> 		at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:1004)
   [junit4]   2> 		at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:640)
   [junit4]   2> 		at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
   [junit4]   2> 		at org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72)
   [junit4]   2> 		at org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264)
   [junit4]   2> 		at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
   [junit4]   2> 		at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
   [junit4]   2> 		at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 	
   [junit4]   2> 1347292 T18404 C3260 oasc.SolrException.log ERROR null:org.eclipse.jetty.io.EofException
   [junit4]   2> 		at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:142)
   [junit4]   2> 		at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:107)
   [junit4]   2> 		at sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:221)
   [junit4]   2> 		at sun.nio.cs.StreamEncoder.implFlushBuffer(StreamEncoder.java:291)
   [junit4]   2> 		at sun.nio.cs.StreamEncoder.implFlush(StreamEncoder.java:295)
   [junit4]   2> 		at sun.nio.cs.StreamEncoder.flush(StreamEncoder.java:141)
   [junit4]   2> 		at java.io.OutputStreamWriter.flush(OutputStreamWriter.java:229)
   [junit4]   2> 		at org.apache.solr.util.FastWriter.flush(FastWriter.java:137)
   [junit4]   2> 		at org.apache.solr.servlet.SolrDispatchFilter.writeResponse(SolrDispatchFilter.java:769)
   [junit4]   2> 		at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:428)
   [junit4]   2> 		at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:208)
   [junit4]   2> 		at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
   [junit4]   2> 		at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:137)
   [junit4]   2> 		at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
   [junit4]   2> 		at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455)
   [junit4]   2> 		at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:229)
   [junit4]   2> 		at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
   [junit4]   2> 		at org.eclipse.jetty.server.handler.GzipHandler.handle(GzipHandler.java:301)
   [junit4]   2> 		at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1077)
   [junit4]   2> 		at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384)
   [junit4]   2> 		at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
   [junit4]   2> 		at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009)
   [junit4]   2> 		at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
   [junit4]   2> 		at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
   [junit4]   2> 		at org.eclipse.jetty.server.Server.handle(Server.java:368)
   [junit4]   2> 		at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489)
   [junit4]   2> 		at org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53)
   [junit4]   2> 		at org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:942)
   [junit4]   2> 		at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:1004)
   [junit4]   2> 		at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:640)
   [junit4]   2> 		at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
   [junit4]   2> 		at org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72)
   [junit4]   2> 		at org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264)
   [junit4]   2> 		at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
   [junit4]   2> 		at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
   [junit4]   2> 		at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 	
   [junit4]   2> 1347294 T18404 C3260 oasc.SolrCore.close [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@52f113a
   [junit4]   2> 1347295 T18404 C3260 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=1,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=64,cumulative_deletesById=0,cumulative_deletesByQuery=1,cumulative_errors=0}
   [junit4]   2> 1347296 T18404 C3260 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
   [junit4]   2> 1347296 T18404 C3260 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
   [junit4]   2> 1347297 T18404 C3260 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
   [junit4]   2> 1347298 T18404 C3260 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
   [junit4]   2> 1347299 T18404 C3260 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
   [junit4]   2> 1347299 T18404 C3260 oasc.CachingDirectoryFactory.closeCacheValue looking to close /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.handler.TestReplicationHandlerBackup-4B12C10A5FB99CCE-001/solr-instance-001/collection1/data [CachedDir<<refCount=0;path=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.handler.TestReplicationHandlerBackup-4B12C10A5FB99CCE-001/solr-instance-001/collection1/data;done=false>>]
   [junit4]   2> 1347300 T18404 C3260 oasc.CachingDirectoryFactory.close Closing directory: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.handler.TestReplicationHandlerBackup-4B12C10A5FB99CCE-001/solr-instance-001/collection1/data
   [junit4]   2> 1347300 T18404 C3260 oasc.CachingDirectoryFactory.closeCacheValue looking to close /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.handler.TestReplicationHandlerBackup-4B12C10A5FB99CCE-001/solr-instance-001/collection1/data/index [CachedDir<<refCount=0;path=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.handler.TestReplicationHandlerBackup-4B12C10A5FB99CCE-001/solr-instance-001/collection1/data/index;done=false>>]
   [junit4]   2> 1347300 T18404 C3260 oasc.CachingDirectoryFactory.close Closing directory: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.handler.TestReplicationHandlerBackup-4B12C10A5FB99CCE-001/solr-instance-001/collection1/data/index
   [junit4]   2> 1352298 T18399 oas.SolrTestCaseJ4.deleteCore ###deleteCore
   [junit4]   2> 7148 T18398 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 1 leaked thread(s).
   [junit4]   2> 27345 T18398 ccr.ThreadLeakControl.checkThreadLeaks SEVERE 1 thread leaked from SUITE scope at org.apache.solr.handler.TestReplicationHandlerBackup: 
   [junit4]   2> 	   1) Thread[id=18419, name=Thread-2467, state=RUNNABLE, group=TGRP-TestReplicationHandlerBackup]
   [junit4]   2> 	        at java.net.PlainSocketImpl.socketConnect(Native Method)
   [junit4]   2> 	        at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
   [junit4]   2> 	        at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
   [junit4]   2> 	        at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
   [junit4]   2> 	        at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
   [junit4]   2> 	        at java.net.Socket.connect(Socket.java:579)
   [junit4]   2> 	        at java.net.Socket.connect(Socket.java:528)
   [junit4]   2> 	        at sun.net.NetworkClient.doConnect(NetworkClient.java:180)
   [junit4]   2> 	        at sun.net.www.http.HttpClient.openServer(HttpClient.java:432)
   [junit4]   2> 	        at sun.net.www.http.HttpClient.openServer(HttpClient.java:527)
   [junit4]   2> 	        at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:652)
   [junit4]   2> 	        at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1323)
   [junit4]   2> 	        at java.net.URL.openStream(URL.java:1037)
   [junit4]   2> 	        at org.apache.solr.handler.TestReplicationHandlerBackup$BackupThread.run(TestReplicationHandlerBackup.java:313)
   [junit4]   2> 27345 T18398 ccr.ThreadLeakControl.tryToInterruptAll Starting to interrupt leaked threads:
   [junit4]   2> 	   1) Thread[id=18419, name=Thread-2467, state=RUNNABLE, group=TGRP-TestReplicationHandlerBackup]
   [junit4]   2> 30353 T18398 ccr.ThreadLeakControl.tryToInterruptAll SEVERE There are still zombie threads that couldn't be terminated:
   [junit4]   2> 	   1) Thread[id=18419, name=Thread-2467, state=RUNNABLE, group=TGRP-TestReplicationHandlerBackup]
   [junit4]   2> 	        at java.net.PlainSocketImpl.socketConnect(Native Method)
   [junit4]   2> 	        at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
   [junit4]   2> 	        at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
   [junit4]   2> 	        at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
   [junit4]   2> 	        at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
   [junit4]   2> 	        at java.net.Socket.connect(Socket.java:579)
   [junit4]   2> 	        at java.net.Socket.connect(Socket.java:528)
   [junit4]   2> 	        at sun.net.NetworkClient.doConnect(NetworkClient.java:180)
   [junit4]   2> 	        at sun.net.www.http.HttpClient.openServer(HttpClient.java:432)
   [junit4]   2> 	        at sun.net.www.http.HttpClient.openServer(HttpClient.java:527)
   [junit4]   2> 	        at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:652)
   [junit4]   2> 	        at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1323)
   [junit4]   2> 	        at java.net.URL.openStream(URL.java:1037)
   [junit4]   2> 	        at org.apache.solr.handler.TestReplicationHandlerBackup$BackupThread.run(TestReplicationHandlerBackup.java:313)
   [junit4]   2> NOTE: test params are: codec=Lucene45, sim=RandomSimilarityProvider(queryNorm=true,coord=crazy): {}, locale=cs, timezone=America/Metlakatla
   [junit4]   2> NOTE: FreeBSD 9.1-RELEASE-p3 amd64/Oracle Corporation 1.7.0_60 (64-bit)/cpus=16,threads=2,free=121687168,total=461373440
   [junit4]   2> NOTE: All tests run in this JVM: [TestAddFieldRealTimeGet, CursorPagingTest, SOLR749Test, DistributedFacetPivotLargeTest, AliasIntegrationTest, LegacyHTMLStripCharFilterTest, TestPseudoReturnFields, DistribDocExpirationUpdateProcessorTest, TestCSVResponseWriter, ClusterStateTest, TestCloudInspectUtil, URLClassifyProcessorTest, TestPHPSerializedResponseWriter, TestFiltering, OpenCloseCoreStressTest, TestCollationField, DocValuesTest, CSVRequestHandlerTest, RAMDirectoryFactoryTest, TestDefaultSearchFieldResource, DateFieldTest, TestSweetSpotSimilarityFactory, TestDynamicFieldCollectionResource, TestOmitPositions, TestCloudManagedSchemaConcurrent, TestManagedResource, TestManagedResourceStorage, OverseerRolesTest, AnalysisAfterCoreReloadTest, TestReversedWildcardFilterFactory, TestMergePolicyConfig, TestWriterPerf, AnalysisErrorHandlingTest, DistanceFunctionTest, TestBinaryField, TestLFUCache, LoggingHandlerTest, TestSchemaVersionResource, SolrRequestParserTest, TestReRankQParserPlugin, TestSearcherReuse, TestPerFieldSimilarity, HighlighterMaxOffsetTest, DistributedTermsComponentTest, SuggesterFSTTest, TestFieldTypeCollectionResource, RequestHandlersTest, TestExceedMaxTermLength, TestQueryTypes, TestZkChroot, SolrInfoMBeanTest, MultiTermTest, PrimUtilsTest, TestFileDictionaryLookup, DirectUpdateHandlerOptimizeTest, AutoCommitTest, DistributedDebugComponentTest, SolrXmlInZkTest, TestCopyFieldCollectionResource, TestElisionMultitermQuery, HardAutoCommitTest, TestUniqueKeyFieldResource, SoftAutoCommitTest, TestCollationFieldDocValues, BinaryUpdateRequestHandlerTest, TestManagedStopFilterFactory, TestCollapseQParserPlugin, TestMissingGroups, CopyFieldTest, TestCollectionAPI, TestOrdValues, DistributedQueryElevationComponentTest, TestSchemaSimilarityResource, TestSolrQueryParser, TestReloadAndDeleteDocs, ResourceLoaderTest, TestCharFilters, ChaosMonkeySafeLeaderTest, TestUtils, SliceStateTest, DistributedQueryComponentCustomSortTest, TestIndexingPerformance, ShardRoutingCustomTest, ChangedSchemaMergeTest, TestHighFrequencyDictionaryFactory, TestSimpleQParserPlugin, SchemaVersionSpecificBehaviorTest, PreAnalyzedFieldTest, TestSolrXmlPersistence, HdfsBasicDistributedZk2Test, TestQuerySenderNoQuery, QueryParsingTest, TestIBSimilarityFactory, TestGroupingSearch, TestReplicationHandlerBackup]
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestReplicationHandlerBackup -Dtests.seed=4B12C10A5FB99CCE -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=cs -Dtests.timezone=America/Metlakatla -Dtests.file.encoding=ISO-8859-1
   [junit4] ERROR   0.00s J1 | TestReplicationHandlerBackup (suite) <<<
   [junit4]    > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.handler.TestReplicationHandlerBackup: 
   [junit4]    >    1) Thread[id=18419, name=Thread-2467, state=RUNNABLE, group=TGRP-TestReplicationHandlerBackup]
   [junit4]    >         at java.net.PlainSocketImpl.socketConnect(Native Method)
   [junit4]    >         at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
   [junit4]    >         at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
   [junit4]    >         at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
   [junit4]    >         at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
   [junit4]    >         at java.net.Socket.connect(Socket.java:579)
   [junit4]    >         at java.net.Socket.connect(Socket.java:528)
   [junit4]    >         at sun.net.NetworkClient.doConnect(NetworkClient.java:180)
   [junit4]    >         at sun.net.www.http.HttpClient.openServer(HttpClient.java:432)
   [junit4]    >         at sun.net.www.http.HttpClient.openServer(HttpClient.java:527)
   [junit4]    >         at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:652)
   [junit4]    >         at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1323)
   [junit4]    >         at java.net.URL.openStream(URL.java:1037)
   [junit4]    >         at org.apache.solr.handler.TestReplicationHandlerBackup$BackupThread.run(TestReplicationHandlerBackup.java:313)
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([4B12C10A5FB99CCE]:0)Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   [junit4]    >    1) Thread[id=18419, name=Thread-2467, state=RUNNABLE, group=TGRP-TestReplicationHandlerBackup]
   [junit4]    >         at java.net.PlainSocketImpl.socketConnect(Native Method)
   [junit4]    >         at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
   [junit4]    >         at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
   [junit4]    >         at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
   [junit4]    >         at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
   [junit4]    >         at java.net.Socket.connect(Socket.java:579)
   [junit4]    >         at java.net.Socket.connect(Socket.java:528)
   [junit4]    >         at sun.net.NetworkClient.doConnect(NetworkClient.java:180)
   [junit4]    >         at sun.net.www.http.HttpClient.openServer(HttpClient.java:432)
   [junit4]    >         at sun.net.www.http.HttpClient.openServer(HttpClient.java:527)
   [junit4]    >         at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:652)
   [junit4]    >         at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1323)
   [junit4]    >         at java.net.URL.openStream(URL.java:1037)
   [junit4]    >         at org.apache.solr.handler.TestReplicationHandlerBackup$BackupThread.run(TestReplicationHandlerBackup.java:313)
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([4B12C10A5FB99CCE]:0)
   [junit4] Completed on J1 in 30.41s, 1 test, 2 errors <<< FAILURES!

[...truncated 740 lines...]
BUILD FAILED
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:474: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:454: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:45: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/extra-targets.xml:37: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build.xml:189: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/common-build.xml:496: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/lucene/common-build.xml:1350: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/lucene/common-build.xml:965: There were test failures: 430 suites, 1782 tests, 2 suite-level errors, 1029 ignored (18 assumptions)

Total time: 83 minutes 17 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Sending artifact delta relative to Lucene-Solr-Tests-trunk-Java7 #4806
Archived 3 artifacts
Archive block size is 32768
Received 0 blocks and 89742500 bytes
Compression is 0.0%
Took 17 sec
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure