You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2014/09/19 17:33:01 UTC
[JENKINS] Lucene-Solr-Tests-5.x-Java7 - Build # 2120 - Still
Failing
Build: https://builds.apache.org/job/Lucene-Solr-Tests-5.x-Java7/2120/
All tests passed
Build Log:
[...truncated 60096 lines...]
BUILD FAILED
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/build.xml:491: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/build.xml:78: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build.xml:548: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build.xml:554: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/lucene/common-build.xml:1959: Tidy was unable to process file /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/docs/solr-core/org/apache/solr/core/InitParams.html, 1 returned.
Total time: 93 minutes 51 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Sending artifact delta relative to Lucene-Solr-Tests-5.x-Java7 #2117
Archived 1 artifacts
Archive block size is 32768
Received 0 blocks and 464 bytes
Compression is 0.0%
Took 12 ms
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
[JENKINS] Lucene-Solr-Tests-5.x-Java7 - Build # 2122 - Still
Failing
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-5.x-Java7/2122/
2 tests failed.
FAILED: junit.framework.TestSuite.org.apache.solr.handler.TestReplicationHandlerBackup
Error Message:
1 thread leaked from SUITE scope at org.apache.solr.handler.TestReplicationHandlerBackup: 1) Thread[id=5881, name=Thread-2058, state=RUNNABLE, group=TGRP-TestReplicationHandlerBackup] at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:579) at java.net.Socket.connect(Socket.java:528) at sun.net.NetworkClient.doConnect(NetworkClient.java:180) at sun.net.www.http.HttpClient.openServer(HttpClient.java:432) at sun.net.www.http.HttpClient.openServer(HttpClient.java:527) at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:652) at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1323) at java.net.URL.openStream(URL.java:1037) at org.apache.solr.handler.TestReplicationHandlerBackup$BackupThread.run(TestReplicationHandlerBackup.java:318)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.handler.TestReplicationHandlerBackup:
1) Thread[id=5881, name=Thread-2058, state=RUNNABLE, group=TGRP-TestReplicationHandlerBackup]
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:579)
at java.net.Socket.connect(Socket.java:528)
at sun.net.NetworkClient.doConnect(NetworkClient.java:180)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:432)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:527)
at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:652)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1323)
at java.net.URL.openStream(URL.java:1037)
at org.apache.solr.handler.TestReplicationHandlerBackup$BackupThread.run(TestReplicationHandlerBackup.java:318)
at __randomizedtesting.SeedInfo.seed([A98D47E8931C2C82]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.handler.TestReplicationHandlerBackup
Error Message:
There are still zombie threads that couldn't be terminated: 1) Thread[id=5881, name=Thread-2058, state=RUNNABLE, group=TGRP-TestReplicationHandlerBackup] at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:579) at java.net.Socket.connect(Socket.java:528) at sun.net.NetworkClient.doConnect(NetworkClient.java:180) at sun.net.www.http.HttpClient.openServer(HttpClient.java:432) at sun.net.www.http.HttpClient.openServer(HttpClient.java:527) at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:652) at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1323) at java.net.URL.openStream(URL.java:1037) at org.apache.solr.handler.TestReplicationHandlerBackup$BackupThread.run(TestReplicationHandlerBackup.java:318)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
1) Thread[id=5881, name=Thread-2058, state=RUNNABLE, group=TGRP-TestReplicationHandlerBackup]
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:579)
at java.net.Socket.connect(Socket.java:528)
at sun.net.NetworkClient.doConnect(NetworkClient.java:180)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:432)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:527)
at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:652)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1323)
at java.net.URL.openStream(URL.java:1037)
at org.apache.solr.handler.TestReplicationHandlerBackup$BackupThread.run(TestReplicationHandlerBackup.java:318)
at __randomizedtesting.SeedInfo.seed([A98D47E8931C2C82]:0)
Build Log:
[...truncated 11534 lines...]
[junit4] Suite: org.apache.solr.handler.TestReplicationHandlerBackup
[junit4] 2> Creating dataDir: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandlerBackup-A98D47E8931C2C82-001/init-core-data-001
[junit4] 2> 1623144 T5861 oas.SolrTestCaseJ4.setUp ###Starting doTestBackup
[junit4] 2> 1623169 T5861 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 1623173 T5861 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:17698
[junit4] 2> 1623173 T5861 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 1623174 T5861 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 1623174 T5861 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandlerBackup-A98D47E8931C2C82-001/solr-instance-001
[junit4] 2> 1623174 T5861 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandlerBackup-A98D47E8931C2C82-001/solr-instance-001/'
[junit4] 2> 1623219 T5861 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandlerBackup-A98D47E8931C2C82-001/solr-instance-001/solr.xml
[junit4] 2> 1623310 T5861 oasc.CoreContainer.<init> New CoreContainer 2147255340
[junit4] 2> 1623311 T5861 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandlerBackup-A98D47E8931C2C82-001/solr-instance-001/]
[junit4] 2> 1623312 T5861 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 1623312 T5861 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 1623313 T5861 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 1623313 T5861 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 1623313 T5861 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 1623314 T5861 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 1623314 T5861 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 1623314 T5861 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 1623314 T5861 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 1623315 T5861 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
[junit4] 2> 1623316 T5861 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 1623316 T5861 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 1623316 T5861 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 1623321 T5871 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandlerBackup-A98D47E8931C2C82-001/solr-instance-001/collection1/'
[junit4] 2> 1623364 T5871 oasc.SolrConfig.<init> Using Lucene MatchVersion: 5.0.0
[junit4] 2> 1623403 T5871 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 1623404 T5871 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 1623411 T5871 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 1623430 T5871 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 1623431 T5871 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from instancedir /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandlerBackup-A98D47E8931C2C82-001/solr-instance-001/collection1/
[junit4] 2> 1623432 T5871 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 1623432 T5871 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandlerBackup-A98D47E8931C2C82-001/solr-instance-001/collection1/, dataDir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandlerBackup-A98D47E8931C2C82-001/solr-instance-001/collection1/data/
[junit4] 2> 1623432 T5871 oasc.SolrCore.<init> JMX monitoring not detected for core: collection1
[junit4] 2> 1623433 T5871 oasc.CachingDirectoryFactory.get return new directory for /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandlerBackup-A98D47E8931C2C82-001/solr-instance-001/collection1/data
[junit4] 2> 1623433 T5871 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandlerBackup-A98D47E8931C2C82-001/solr-instance-001/collection1/data/index/
[junit4] 2> 1623433 T5871 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandlerBackup-A98D47E8931C2C82-001/solr-instance-001/collection1/data/index' doesn't exist. Creating new index...
[junit4] 2> 1623434 T5871 oasc.CachingDirectoryFactory.get return new directory for /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandlerBackup-A98D47E8931C2C82-001/solr-instance-001/collection1/data/index
[junit4] 2> 1623435 T5871 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@33eb4a24
[junit4] 2> 1623435 T5871 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory(id=lucene-26ed753a)),segFN=segments_1,generation=1}
[junit4] 2> 1623436 T5871 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 1623437 T5871 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 1623437 T5871 oasc.RequestHandlers.initHandlersFromConfig created /update/json: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 1623437 T5871 oasc.RequestHandlers.initHandlersFromConfig created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 1623438 T5871 oasc.RequestHandlers.initHandlersFromConfig created /update/csv: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 1623438 T5871 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 1623438 T5871 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 1623439 T5871 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 1623457 T5871 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 1623459 T5871 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 1623461 T5871 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 1623463 T5871 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 1623465 T5871 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 1623465 T5871 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 1623466 T5871 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=15, maxMergeAtOnceExplicit=39, maxMergedSegmentMB=80.3427734375, floorSegmentMB=2.0517578125, forceMergeDeletesPctAllowed=15.211227451170494, segmentsPerTier=48.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.5411852219002206
[junit4] 2> 1623466 T5871 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory(id=lucene-26ed753a)),segFN=segments_1,generation=1}
[junit4] 2> 1623466 T5871 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 1623467 T5871 oass.SolrIndexSearcher.<init> Opening Searcher@a672a92[collection1] main
[junit4] 2> 1623467 T5871 oasr.ManagedResourceStorage$FileStorageIO.configure File-based storage initialized to use dir: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandlerBackup-A98D47E8931C2C82-001/solr-instance-001/collection1/conf
[junit4] 2> 1623467 T5871 oasr.RestManager.init Initializing RestManager with initArgs: {storageDir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandlerBackup-A98D47E8931C2C82-001/solr-instance-001/collection1/conf}
[junit4] 2> 1623468 T5871 oasr.ManagedResourceStorage.load Reading _rest_managed.json using file:dir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandlerBackup-A98D47E8931C2C82-001/solr-instance-001/collection1/conf
[junit4] 2> 1623469 T5871 oasr.ManagedResource.reloadFromStorage WARN No stored data found for /rest/managed
[junit4] 2> 1623469 T5871 oasr.ManagedResourceStorage$JsonStorage.store Saved JSON object to path _rest_managed.json using file:dir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandlerBackup-A98D47E8931C2C82-001/solr-instance-001/collection1/conf
[junit4] 2> 1623469 T5871 oasr.RestManager.init Initializing 0 registered ManagedResources
[junit4] 2> 1623470 T5871 oash.ReplicationHandler.inform Replication enabled for following config files: schema-replication2.xml:schema.xml
[junit4] 2> 1623470 T5871 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4] 2> 1623470 T5872 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@a672a92[collection1] main{UninvertingDirectoryReader()}
[junit4] 2> 1623470 T5871 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 1623471 T5861 oass.SolrDispatchFilter.init user.dir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2
[junit4] 2> 1623472 T5861 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> ASYNC NEW_CORE C684 name=collection1 org.apache.solr.core.SolrCore@d114b1e
[junit4] 2> 1623474 T5864 C684 oasu.DirectUpdateHandler2.deleteAll [collection1] REMOVING ALL DOCUMENTS FROM INDEX
[junit4] 2> 1623475 T5864 C684 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {deleteByQuery=*:*} 0 1
[junit4] 2> 1623478 T5864 C684 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[0]} 0 0
[junit4] 2> 1623481 T5864 C684 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[1]} 0 0
[junit4] 2> 1623482 T5864 C684 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[2]} 0 0
[junit4] 2> 1623484 T5864 C684 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[3]} 0 0
[junit4] 2> 1623486 T5864 C684 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[4]} 0 0
[junit4] 2> 1623489 T5864 C684 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[5]} 0 0
[junit4] 2> 1623492 T5864 C684 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[6]} 0 0
[junit4] 2> 1623494 T5864 C684 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[7]} 0 0
[junit4] 2> 1623496 T5864 C684 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[8]} 0 0
[junit4] 2> 1623498 T5864 C684 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[9]} 0 0
[junit4] 2> 1623500 T5864 C684 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[10]} 0 0
[junit4] 2> 1623503 T5864 C684 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[11]} 0 0
[junit4] 2> 1623505 T5864 C684 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[12]} 0 0
[junit4] 2> 1623507 T5864 C684 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[13]} 0 0
[junit4] 2> 1623509 T5864 C684 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[14]} 0 0
[junit4] 2> 1623512 T5864 C684 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[15]} 0 0
[junit4] 2> 1623514 T5864 C684 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[16]} 0 0
[junit4] 2> 1623516 T5864 C684 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[17]} 0 0
[junit4] 2> 1623518 T5864 C684 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[18]} 0 0
[junit4] 2> 1623520 T5864 C684 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[19]} 0 0
[junit4] 2> 1623522 T5864 C684 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[20]} 0 0
[junit4] 2> 1623524 T5864 C684 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[21]} 0 0
[junit4] 2> 1623526 T5864 C684 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[22]} 0 0
[junit4] 2> 1623529 T5864 C684 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[23]} 0 0
[junit4] 2> 1623531 T5864 C684 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[24]} 0 0
[junit4] 2> 1623533 T5864 C684 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[25]} 0 0
[junit4] 2> 1623535 T5864 C684 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[26]} 0 0
[junit4] 2> 1623537 T5864 C684 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 1623539 T5864 C684 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory(id=lucene-26ed753a)),segFN=segments_1,generation=1}
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory(id=lucene-26ed753a)),segFN=segments_2,generation=2}
[junit4] 2> 1623540 T5864 C684 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4] 2> 1623541 T5864 C684 oass.SolrIndexSearcher.<init> Opening Searcher@4816f010[collection1] main
[junit4] 2> 1623541 T5864 C684 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4] 2> 1623541 T5872 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4816f010[collection1] main{UninvertingDirectoryReader(Uninverting(_0(5.0.0):C27))}
[junit4] 2> 1623542 T5864 C684 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={softCommit=false&version=2&commit=true&wt=javabin&waitSearcher=true} {commit=} 0 5
[junit4] 2> 1623547 T5866 C684 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1
[junit4] 2> 1623576 T5865 C684 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=backup&name=swmwbazyxlqjub} status=0 QTime=30
[junit4] 2> 1623576 T5876 oash.SnapShooter.createSnapshot Creating backup snapshot...
[junit4] 2> 1623592 T5876 oash.SnapShooter.createSnapshot Done creating backup snapshot: swmwbazyxlqjub
[junit4] 2> 1623751 T5865 C684 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1
[junit4] 2> 1624761 T5865 C684 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1
[junit4] 2> 1624762 T5866 C684 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=backup&name=owarlxwbuujcz} status=0 QTime=2
[junit4] 2> 1624762 T5878 oash.SnapShooter.createSnapshot Creating backup snapshot...
[junit4] 2> 1624769 T5878 oash.SnapShooter.createSnapshot Done creating backup snapshot: owarlxwbuujcz
[junit4] 2> 1624963 T5866 C684 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0
[junit4] 2> 1624970 T5865 C684 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0
[junit4] 2> 1624973 T5866 C684 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=deletebackup&name=swmwbazyxlqjub} status=0 QTime=4
[junit4] 2> 1624973 T5880 oash.SnapShooter.deleteNamedSnapshot Deleting snapshot: swmwbazyxlqjub
[junit4] 2> 1625172 T5866 C684 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0
[junit4] 2> 1625174 T5866 C684 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0
[junit4] 2> 1625175 T5861 oas.SolrTestCaseJ4.tearDown ###Ending doTestBackup
[junit4] 2> 1625175 T5882 oash.SnapShooter.deleteNamedSnapshot Deleting snapshot: owarlxwbuujcz
[junit4] 2> 1625175 T5865 C684 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=deletebackup&name=owarlxwbuujcz} status=0 QTime=1
[junit4] 2> 1625175 T5861 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=2147255340
[junit4] 2> 1625176 T5861 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/solr,null}
[junit4] 2> 1625176 T5865 C684 oasc.SolrException.log ERROR null:org.eclipse.jetty.io.EofException
[junit4] 2> at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:142)
[junit4] 2> at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:107)
[junit4] 2> at sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:221)
[junit4] 2> at sun.nio.cs.StreamEncoder.implFlushBuffer(StreamEncoder.java:291)
[junit4] 2> at sun.nio.cs.StreamEncoder.implFlush(StreamEncoder.java:295)
[junit4] 2> at sun.nio.cs.StreamEncoder.flush(StreamEncoder.java:141)
[junit4] 2> at java.io.OutputStreamWriter.flush(OutputStreamWriter.java:229)
[junit4] 2> at org.apache.solr.util.FastWriter.flush(FastWriter.java:137)
[junit4] 2> at org.apache.solr.response.QueryResponseWriterUtil.writeQueryResponse(QueryResponseWriterUtil.java:54)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.writeResponse(SolrDispatchFilter.java:771)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:424)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:202)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
[junit4] 2> at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:137)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:229)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
[junit4] 2> at org.eclipse.jetty.server.handler.GzipHandler.handle(GzipHandler.java:301)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1077)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
[junit4] 2> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
[junit4] 2> at org.eclipse.jetty.server.Server.handle(Server.java:368)
[junit4] 2> at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489)
[junit4] 2> at org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53)
[junit4] 2> at org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:942)
[junit4] 2> at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:1004)
[junit4] 2> at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:640)
[junit4] 2> at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
[junit4] 2> at org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72)
[junit4] 2> at org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2>
[junit4] 2> 1625178 T5865 C684 oasc.SolrException.log ERROR null:org.eclipse.jetty.io.EofException
[junit4] 2> at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:142)
[junit4] 2> at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:107)
[junit4] 2> at sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:221)
[junit4] 2> at sun.nio.cs.StreamEncoder.implFlushBuffer(StreamEncoder.java:291)
[junit4] 2> at sun.nio.cs.StreamEncoder.implFlush(StreamEncoder.java:295)
[junit4] 2> at sun.nio.cs.StreamEncoder.flush(StreamEncoder.java:141)
[junit4] 2> at java.io.OutputStreamWriter.flush(OutputStreamWriter.java:229)
[junit4] 2> at org.apache.solr.util.FastWriter.flush(FastWriter.java:137)
[junit4] 2> at org.apache.solr.response.QueryResponseWriterUtil.writeQueryResponse(QueryResponseWriterUtil.java:54)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.writeResponse(SolrDispatchFilter.java:771)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:424)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:202)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
[junit4] 2> at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:137)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:229)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
[junit4] 2> at org.eclipse.jetty.server.handler.GzipHandler.handle(GzipHandler.java:301)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1077)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
[junit4] 2> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
[junit4] 2> at org.eclipse.jetty.server.Server.handle(Server.java:368)
[junit4] 2> at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489)
[junit4] 2> at org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53)
[junit4] 2> at org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:942)
[junit4] 2> at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:1004)
[junit4] 2> at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:640)
[junit4] 2> at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
[junit4] 2> at org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72)
[junit4] 2> at org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2>
[junit4] 2> 1625194 T5865 C684 oasc.SolrCore.close [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@d114b1e
[junit4] 2> 1625194 T5865 C684 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=1,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=27,cumulative_deletesById=0,cumulative_deletesByQuery=1,cumulative_errors=0}
[junit4] 2> 1625195 T5865 C684 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4] 2> 1625195 T5865 C684 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4] 2> 1625195 T5865 C684 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4] 2> 1625196 T5865 C684 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4] 2> 1625196 T5865 C684 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
[junit4] 2> 1625196 T5865 C684 oasc.CachingDirectoryFactory.closeCacheValue looking to close /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandlerBackup-A98D47E8931C2C82-001/solr-instance-001/collection1/data [CachedDir<<refCount=0;path=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandlerBackup-A98D47E8931C2C82-001/solr-instance-001/collection1/data;done=false>>]
[junit4] 2> 1625197 T5865 C684 oasc.CachingDirectoryFactory.close Closing directory: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandlerBackup-A98D47E8931C2C82-001/solr-instance-001/collection1/data
[junit4] 2> 1625197 T5865 C684 oasc.CachingDirectoryFactory.closeCacheValue looking to close /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandlerBackup-A98D47E8931C2C82-001/solr-instance-001/collection1/data/index [CachedDir<<refCount=0;path=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandlerBackup-A98D47E8931C2C82-001/solr-instance-001/collection1/data/index;done=false>>]
[junit4] 2> 1625197 T5865 C684 oasc.CachingDirectoryFactory.close Closing directory: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandlerBackup-A98D47E8931C2C82-001/solr-instance-001/collection1/data/index
[junit4] 2> 1630185 T5861 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4] 2> 7051 T5860 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> 27171 T5860 ccr.ThreadLeakControl.checkThreadLeaks SEVERE 1 thread leaked from SUITE scope at org.apache.solr.handler.TestReplicationHandlerBackup:
[junit4] 2> 1) Thread[id=5881, name=Thread-2058, state=RUNNABLE, group=TGRP-TestReplicationHandlerBackup]
[junit4] 2> at java.net.PlainSocketImpl.socketConnect(Native Method)
[junit4] 2> at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
[junit4] 2> at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
[junit4] 2> at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
[junit4] 2> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
[junit4] 2> at java.net.Socket.connect(Socket.java:579)
[junit4] 2> at java.net.Socket.connect(Socket.java:528)
[junit4] 2> at sun.net.NetworkClient.doConnect(NetworkClient.java:180)
[junit4] 2> at sun.net.www.http.HttpClient.openServer(HttpClient.java:432)
[junit4] 2> at sun.net.www.http.HttpClient.openServer(HttpClient.java:527)
[junit4] 2> at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:652)
[junit4] 2> at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1323)
[junit4] 2> at java.net.URL.openStream(URL.java:1037)
[junit4] 2> at org.apache.solr.handler.TestReplicationHandlerBackup$BackupThread.run(TestReplicationHandlerBackup.java:318)
[junit4] 2> 27172 T5860 ccr.ThreadLeakControl.tryToInterruptAll Starting to interrupt leaked threads:
[junit4] 2> 1) Thread[id=5881, name=Thread-2058, state=RUNNABLE, group=TGRP-TestReplicationHandlerBackup]
[junit4] 2> 30178 T5860 ccr.ThreadLeakControl.tryToInterruptAll SEVERE There are still zombie threads that couldn't be terminated:
[junit4] 2> 1) Thread[id=5881, name=Thread-2058, state=RUNNABLE, group=TGRP-TestReplicationHandlerBackup]
[junit4] 2> at java.net.PlainSocketImpl.socketConnect(Native Method)
[junit4] 2> at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
[junit4] 2> at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
[junit4] 2> at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
[junit4] 2> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
[junit4] 2> at java.net.Socket.connect(Socket.java:579)
[junit4] 2> at java.net.Socket.connect(Socket.java:528)
[junit4] 2> at sun.net.NetworkClient.doConnect(NetworkClient.java:180)
[junit4] 2> at sun.net.www.http.HttpClient.openServer(HttpClient.java:432)
[junit4] 2> at sun.net.www.http.HttpClient.openServer(HttpClient.java:527)
[junit4] 2> at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:652)
[junit4] 2> at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1323)
[junit4] 2> at java.net.URL.openStream(URL.java:1037)
[junit4] 2> at org.apache.solr.handler.TestReplicationHandlerBackup$BackupThread.run(TestReplicationHandlerBackup.java:318)
[junit4] 2> NOTE: test params are: codec=Lucene410: {name=PostingsFormat(name=Memory doPackFST= true), id=PostingsFormat(name=Memory doPackFST= true)}, docValues:{}, sim=RandomSimilarityProvider(queryNorm=false,coord=crazy): {}, locale=es_PR, timezone=America/Mendoza
[junit4] 2> NOTE: FreeBSD 9.1-RELEASE-p3 amd64/Oracle Corporation 1.7.0_65 (64-bit)/cpus=16,threads=2,free=155714032,total=461373440
[junit4] 2> NOTE: All tests run in this JVM: [TestSearcherReuse, TestSearchPerf, TestCopyFieldCollectionResource, ExternalCollectionsTest, TestDefaultSearchFieldResource, TestRecoveryHdfs, TestLazyCores, StatsComponentTest, TermVectorComponentTest, SampleTest, AtomicUpdatesTest, FastVectorHighlighterTest, TestCoreContainer, LukeRequestHandlerTest, BJQParserTest, VMParamsZkACLAndCredentialsProvidersTest, AnalysisAfterCoreReloadTest, TestMiniSolrCloudCluster, StandardRequestHandlerTest, OverriddenZkACLAndCredentialsProvidersTest, TestIBSimilarityFactory, UpdateRequestProcessorFactoryTest, TestOmitPositions, HttpPartitionTest, TestCollationFieldDocValues, PingRequestHandlerTest, BlockCacheTest, TestCursorMarkWithoutUniqueKey, TestDistributedMissingSort, PeerSyncTest, CursorPagingTest, BasicDistributedZkTest, TestLRUCache, AddBlockUpdateTest, SyncSliceTest, SolrRequestParserTest, SpellPossibilityIteratorTest, DistanceFunctionTest, TestFuzzyAnalyzedSuggestions, OpenExchangeRatesOrgProviderTest, TestLeaderElectionZkExpiry, CoreAdminCreateDiscoverTest, TestComponentsName, TestNRTOpen, DirectUpdateHandlerTest, PreAnalyzedFieldTest, IndexSchemaRuntimeFieldTest, TestFileDictionaryLookup, TestSolrDeletionPolicy2, TestRandomMergePolicy, TestInfoStreamLogging, TestSort, ConnectionManagerTest, DistributedSpellCheckComponentTest, BasicZkTest, DistribCursorPagingTest, DistributedIntervalFacetingTest, TestManagedSchemaFieldResource, ChaosMonkeyNothingIsSafeTest, TestModifyConfFiles, TestRandomDVFaceting, DistributedFacetPivotLargeTest, ZkSolrClientTest, DistributedQueryComponentCustomSortTest, TestJoin, ShardRoutingCustomTest, SoftAutoCommitTest, SolrCoreTest, CoreAdminHandlerTest, TestReplicationHandlerBackup]
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestReplicationHandlerBackup -Dtests.seed=A98D47E8931C2C82 -Dtests.multiplier=2 -Dtests.slow=true -Dtests.locale=es_PR -Dtests.timezone=America/Mendoza -Dtests.file.encoding=US-ASCII
[junit4] ERROR 0.00s J2 | TestReplicationHandlerBackup (suite) <<<
[junit4] > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.handler.TestReplicationHandlerBackup:
[junit4] > 1) Thread[id=5881, name=Thread-2058, state=RUNNABLE, group=TGRP-TestReplicationHandlerBackup]
[junit4] > at java.net.PlainSocketImpl.socketConnect(Native Method)
[junit4] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
[junit4] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
[junit4] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
[junit4] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
[junit4] > at java.net.Socket.connect(Socket.java:579)
[junit4] > at java.net.Socket.connect(Socket.java:528)
[junit4] > at sun.net.NetworkClient.doConnect(NetworkClient.java:180)
[junit4] > at sun.net.www.http.HttpClient.openServer(HttpClient.java:432)
[junit4] > at sun.net.www.http.HttpClient.openServer(HttpClient.java:527)
[junit4] > at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:652)
[junit4] > at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1323)
[junit4] > at java.net.URL.openStream(URL.java:1037)
[junit4] > at org.apache.solr.handler.TestReplicationHandlerBackup$BackupThread.run(TestReplicationHandlerBackup.java:318)
[junit4] > at __randomizedtesting.SeedInfo.seed([A98D47E8931C2C82]:0)Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
[junit4] > 1) Thread[id=5881, name=Thread-2058, state=RUNNABLE, group=TGRP-TestReplicationHandlerBackup]
[junit4] > at java.net.PlainSocketImpl.socketConnect(Native Method)
[junit4] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
[junit4] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
[junit4] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
[junit4] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
[junit4] > at java.net.Socket.connect(Socket.java:579)
[junit4] > at java.net.Socket.connect(Socket.java:528)
[junit4] > at sun.net.NetworkClient.doConnect(NetworkClient.java:180)
[junit4] > at sun.net.www.http.HttpClient.openServer(HttpClient.java:432)
[junit4] > at sun.net.www.http.HttpClient.openServer(HttpClient.java:527)
[junit4] > at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:652)
[junit4] > at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1323)
[junit4] > at java.net.URL.openStream(URL.java:1037)
[junit4] > at org.apache.solr.handler.TestReplicationHandlerBackup$BackupThread.run(TestReplicationHandlerBackup.java:318)
[junit4] > at __randomizedtesting.SeedInfo.seed([A98D47E8931C2C82]:0)
[junit4] Completed on J2 in 30.24s, 1 test, 2 errors <<< FAILURES!
[...truncated 429 lines...]
BUILD FAILED
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/build.xml:491: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/build.xml:471: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/build.xml:60: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/extra-targets.xml:39: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build.xml:189: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/common-build.xml:496: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/lucene/common-build.xml:1359: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/lucene/common-build.xml:962: There were test failures: 429 suites, 1772 tests, 2 suite-level errors, 515 ignored (22 assumptions)
Total time: 50 minutes 16 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Sending artifact delta relative to Lucene-Solr-Tests-5.x-Java7 #2117
Archived 5 artifacts
Archive block size is 32768
Received 0 blocks and 147941425 bytes
Compression is 0.0%
Took 27 sec
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
[JENKINS] Lucene-Solr-Tests-5.x-Java7 - Build # 2121 - Still
Failing
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-5.x-Java7/2121/
1 tests failed.
REGRESSION: org.apache.solr.cloud.ChaosMonkeySafeLeaderTest.testDistribSearch
Error Message:
Stack Trace:
java.lang.NullPointerException
at __randomizedtesting.SeedInfo.seed([D1E9884D0A34B708:500F06557D6BD734]:0)
at org.apache.solr.client.solrj.impl.CloudSolrServer.requestWithRetryOnStaleState(CloudSolrServer.java:706)
at org.apache.solr.client.solrj.impl.CloudSolrServer.request(CloudSolrServer.java:601)
at org.apache.solr.client.solrj.request.AbstractUpdateRequest.process(AbstractUpdateRequest.java:124)
at org.apache.solr.client.solrj.SolrServer.deleteByQuery(SolrServer.java:285)
at org.apache.solr.client.solrj.SolrServer.deleteByQuery(SolrServer.java:271)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.del(AbstractFullDistribZkTestBase.java:729)
at org.apache.solr.cloud.ChaosMonkeySafeLeaderTest.tryDelete(ChaosMonkeySafeLeaderTest.java:194)
at org.apache.solr.cloud.ChaosMonkeySafeLeaderTest.doTest(ChaosMonkeySafeLeaderTest.java:112)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:869)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
at java.lang.Thread.run(Thread.java:745)
Build Log:
[...truncated 11272 lines...]
[junit4] Suite: org.apache.solr.cloud.ChaosMonkeySafeLeaderTest
[junit4] 2> Creating dataDir: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.ChaosMonkeySafeLeaderTest-D1E9884D0A34B708-001/init-core-data-001
[junit4] 2> 1117621 T3965 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (true) and clientAuth (false)
[junit4] 2> 1117621 T3965 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
[junit4] 2> 1117626 T3965 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4] 2> 1117626 T3965 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 1117627 T3966 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 1117727 T3965 oasc.ZkTestServer.run start zk server on port:12240
[junit4] 2> 1117728 T3965 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
[junit4] 2> 1117728 T3965 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1117732 T3972 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@630ba13e name:ZooKeeperConnection Watcher:127.0.0.1:12240 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1117732 T3965 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1117732 T3965 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
[junit4] 2> 1117733 T3965 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 1117735 T3965 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
[junit4] 2> 1117736 T3965 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1117737 T3974 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@759bb4e6 name:ZooKeeperConnection Watcher:127.0.0.1:12240/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1117737 T3965 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1117738 T3965 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
[junit4] 2> 1117738 T3965 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4] 2> 1117740 T3965 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4] 2> 1117741 T3965 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4] 2> 1117742 T3965 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4] 2> 1117743 T3965 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 1117743 T3965 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4] 2> 1117745 T3965 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/core/src/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
[junit4] 2> 1117746 T3965 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4] 2> 1117847 T3965 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 1117847 T3965 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 1117849 T3965 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 1117849 T3965 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4] 2> 1117851 T3965 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 1117851 T3965 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4] 2> 1117853 T3965 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 1117853 T3965 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4] 2> 1117855 T3965 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 1117855 T3965 oascc.SolrZkClient.makePath makePath: /configs/conf1/enumsConfig.xml
[junit4] 2> 1117857 T3965 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 1117857 T3965 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4] 2> 1117858 T3965 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 1117859 T3965 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 1117860 T3965 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 1117861 T3965 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4] 2> 1117862 T3965 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 1117863 T3965 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4] 2> 1117865 T3965 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
[junit4] 2> 1117866 T3965 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1117867 T3976 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@339767eb name:ZooKeeperConnection Watcher:127.0.0.1:12240/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1117867 T3965 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1117867 T3965 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
[junit4] 2> 1119405 T3965 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 1119411 T3965 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
[junit4] 2> 1119415 T3965 oejs.AbstractConnector.doStart Started SslSocketConnector@127.0.0.1:12249
[junit4] 2> 1119418 T3965 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 1119419 T3965 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 1119419 T3965 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.ChaosMonkeySafeLeaderTest-D1E9884D0A34B708-001/tempDir-002
[junit4] 2> 1119419 T3965 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.ChaosMonkeySafeLeaderTest-D1E9884D0A34B708-001/tempDir-002/'
[junit4] 2> 1119477 T3965 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J3/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.ChaosMonkeySafeLeaderTest-D1E9884D0A34B708-001/tempDir-002/solr.xml
[junit4] 2> 1119581 T3965 oasc.CoreContainer.<init> New CoreContainer 175473542
[junit4] 2> 1119582 T3965 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.ChaosMonkeySafeLeaderTest-D1E9884D0A34B708-001/tempDir-002/]
[junit4] 2> 1119583 T3965 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 1119583 T3965 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 1119584 T3965 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 1119584 T3965 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 1119584 T3965 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 1119585 T3965 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 1119585 T3965 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 1119585 T3965 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 1119586 T3965 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 1119589 T3965 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
[junit4] 2> 1119591 T3965 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 1119591 T3965 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 1119592 T3965 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 1119592 T3965 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:12240/solr
[junit4] 2> 1119592 T3965 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 1119593 T3965 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
[junit4] 2> 1119594 T3965 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1119595 T3987 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1aea11e6 name:ZooKeeperConnection Watcher:127.0.0.1:12240 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1119596 T3965 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1119597 T3965 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
[junit4] 2> 1119599 T3965 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1119600 T3989 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@fb18bbb name:ZooKeeperConnection Watcher:127.0.0.1:12240/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1119601 T3965 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1119602 T3965 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 1119604 T3965 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 1119606 T3965 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
[junit4] 2> 1119608 T3965 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
[junit4] 2> 1119610 T3965 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
[junit4] 2> 1119612 T3965 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 1119614 T3965 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:12249_
[junit4] 2> 1119614 T3965 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:12249_
[junit4] 2> 1119616 T3965 oascc.SolrZkClient.makePath makePath: /overseer_elect
[junit4] 2> 1119617 T3965 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 1119619 T3965 oasc.Overseer.close Overseer (id=null) closing
[junit4] 2> 1119620 T3965 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:12249_
[junit4] 2> 1119620 T3965 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 1119622 T3965 oasc.Overseer.start Overseer (id=92481629410689028-127.0.0.1:12249_-n_0000000000) starting
[junit4] 2> 1119624 T3965 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 1119630 T3965 oasc.OverseerAutoReplicaFailoverThread.<init> Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
[junit4] 2> 1119630 T3991 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 1119632 T3965 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 1119633 T3965 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 1119635 T3965 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1119637 T3990 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 1119644 T3993 oasc.ZkController.publish publishing core=collection1 state=down collection=control_collection
[junit4] 2> 1119644 T3993 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 1119645 T3993 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 1119645 T3994 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1119647 T3990 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "core":"collection1",
[junit4] 2> "base_url":"https://127.0.0.1:12249",
[junit4] 2> "node_name":"127.0.0.1:12249_",
[junit4] 2> "roles":null,
[junit4] 2> "shard":null,
[junit4] 2> "numShards":"1",
[junit4] 2> "collection":"control_collection",
[junit4] 2> "state":"down"}
[junit4] 2> 1119647 T3990 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with shards [shard1]
[junit4] 2> 1119648 T3990 oasc.Overseer$ClusterStateUpdater.createCollection state version control_collection 1
[junit4] 2> 1119648 T3990 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 1119650 T3994 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1120645 T3993 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 1120646 T3993 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4] 2> 1120646 T3993 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 1120647 T3993 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
[junit4] 2> 1120647 T3993 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 1120647 T3993 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.ChaosMonkeySafeLeaderTest-D1E9884D0A34B708-001/tempDir-002/collection1/'
[junit4] 2> 1120650 T3993 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.ChaosMonkeySafeLeaderTest-D1E9884D0A34B708-001/tempDir-002/collection1/lib/classes/' to classloader
[junit4] 2> 1120651 T3993 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.ChaosMonkeySafeLeaderTest-D1E9884D0A34B708-001/tempDir-002/collection1/lib/.svn/' to classloader
[junit4] 2> 1120652 T3993 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.ChaosMonkeySafeLeaderTest-D1E9884D0A34B708-001/tempDir-002/collection1/lib/README' to classloader
[junit4] 2> 1120715 T3993 oasc.SolrConfig.<init> Using Lucene MatchVersion: 5.0.0
[junit4] 2> 1120760 T3993 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 1120861 T3993 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 1120889 T3993 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 1121212 T3993 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 1121214 T3993 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 1121216 T3993 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 1121221 T3993 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 1121236 T3993 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection control_collection
[junit4] 2> 1121237 T3993 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4] 2> 1121302 T3993 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.ChaosMonkeySafeLeaderTest-D1E9884D0A34B708-001/tempDir-002/collection1/, dataDir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.ChaosMonkeySafeLeaderTest-D1E9884D0A34B708-001/tempDir-001/control/data/
[junit4] 2> 1121302 T3993 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@66bf378a
[junit4] 2> 1121305 T3993 oasc.CachingDirectoryFactory.get return new directory for /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.ChaosMonkeySafeLeaderTest-D1E9884D0A34B708-001/tempDir-001/control/data
[junit4] 2> 1121306 T3993 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.ChaosMonkeySafeLeaderTest-D1E9884D0A34B708-001/tempDir-001/control/data/index/
[junit4] 2> 1121306 T3993 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.ChaosMonkeySafeLeaderTest-D1E9884D0A34B708-001/tempDir-001/control/data/index' doesn't exist. Creating new index...
[junit4] 2> 1121307 T3993 oasc.CachingDirectoryFactory.get return new directory for /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.ChaosMonkeySafeLeaderTest-D1E9884D0A34B708-001/tempDir-001/control/data/index
[junit4] 2> 1121308 T3993 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=31, maxMergeAtOnceExplicit=22, maxMergedSegmentMB=93.6494140625, floorSegmentMB=0.2197265625, forceMergeDeletesPctAllowed=13.12195925824124, segmentsPerTier=23.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> 1121332 T3993 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.ChaosMonkeySafeLeaderTest-D1E9884D0A34B708-001/tempDir-001/control/data/index,segFN=segments_1,generation=1}
[junit4] 2> 1121332 T3993 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 1121337 T3993 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 1121337 T3993 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 1121337 T3993 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 1121338 T3993 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 1121338 T3993 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 1121338 T3993 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 1121339 T3993 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 1121339 T3993 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 1121339 T3993 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 1121340 T3993 oasc.RequestHandlers.initHandlersFromConfig created /update/json: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 1121340 T3993 oasc.RequestHandlers.initHandlersFromConfig created /update/csv: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 1121341 T3993 oasc.RequestHandlers.initHandlersFromConfig created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 1121341 T3993 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 1121342 T3993 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 1121342 T3993 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 1121342 T3993 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 1121342 T3993 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 1121343 T3993 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 1121343 T3993 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 1121344 T3993 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
[junit4] 2> 1121362 T3993 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 1121364 T3993 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 1121366 T3993 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 1121368 T3993 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 1121371 T3993 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 1121371 T3993 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 1121372 T3993 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=13, maxMergeAtOnceExplicit=27, maxMergedSegmentMB=68.30859375, floorSegmentMB=1.7421875, forceMergeDeletesPctAllowed=13.621698894646508, segmentsPerTier=28.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> 1121374 T3993 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.ChaosMonkeySafeLeaderTest-D1E9884D0A34B708-001/tempDir-001/control/data/index,segFN=segments_1,generation=1}
[junit4] 2> 1121374 T3993 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 1121375 T3993 oass.SolrIndexSearcher.<init> Opening Searcher@4eeadc8a[collection1] main
[junit4] 2> 1121375 T3993 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
[junit4] 2> 1121376 T3993 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 1121376 T3993 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
[junit4] 2> 1121376 T3993 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 1121377 T3993 oasr.RestManager.init Initializing RestManager with initArgs: {}
[junit4] 2> 1121377 T3993 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1121377 T3993 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json
[junit4] 2> 1121378 T3993 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1121378 T3993 oascc.SolrZkClient.makePath makePath: /configs/conf1/_rest_managed.json
[junit4] 2> 1121380 T3993 oasr.ManagedResourceStorage$ZooKeeperStorageIO$1.close Wrote 38 bytes to new znode /configs/conf1/_rest_managed.json
[junit4] 2> 1121380 T3993 oasr.ManagedResourceStorage$JsonStorage.store Saved JSON object to path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1121380 T3993 oasr.RestManager.init Initializing 0 registered ManagedResources
[junit4] 2> 1121383 T3995 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4eeadc8a[collection1] main{UninvertingDirectoryReader()}
[junit4] 2> 1121386 T3993 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 1121387 T3998 oasc.ZkController.register Register replica - core:collection1 address:https://127.0.0.1:12249 collection:control_collection shard:shard1
[junit4] 2> 1121387 T3965 oass.SolrDispatchFilter.init user.dir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J3
[junit4] 2> 1121388 T3965 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 1121388 T3998 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4] 2> 1121392 T3998 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 1121393 T3965 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
[junit4] 2> 1121393 T3994 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1121393 T3998 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 1121394 T3998 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> 1121394 T3965 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> ASYNC NEW_CORE C384 name=collection1 org.apache.solr.core.SolrCore@1c87ab4 url=https://127.0.0.1:12249/collection1 node=127.0.0.1:12249_ C384_STATE=coll:control_collection core:collection1 props:{core=collection1, base_url=https://127.0.0.1:12249, node_name=127.0.0.1:12249_, state=down}
[junit4] 2> 1121394 T3998 C384 P12249 oasc.SyncStrategy.sync Sync replicas to https://127.0.0.1:12249/collection1/
[junit4] 2> 1121395 T3998 C384 P12249 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 1121395 T3998 C384 P12249 oasc.SyncStrategy.syncToMe https://127.0.0.1:12249/collection1/ has no replicas
[junit4] 2> 1121395 T3998 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: https://127.0.0.1:12249/collection1/ shard1
[junit4] 2> 1121395 T4000 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3ad445f0 name:ZooKeeperConnection Watcher:127.0.0.1:12240/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1121395 T3998 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4] 2> 1121395 T3965 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1121396 T3965 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
[junit4] 2> 1121398 T3965 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1121398 T3994 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1121400 T3965 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 1121403 T3994 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1121404 T3994 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1121404 T4001 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1121451 T3998 oasc.ZkController.register We are https://127.0.0.1:12249/collection1/ and leader is https://127.0.0.1:12249/collection1/
[junit4] 2> 1121452 T3998 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:12249
[junit4] 2> 1121452 T3998 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 1121452 T3998 oasc.ZkController.publish publishing core=collection1 state=active collection=control_collection
[junit4] 2> 1121452 T3998 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 1121453 T3994 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1121456 T3990 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "core":"collection1",
[junit4] 2> "base_url":"https://127.0.0.1:12249",
[junit4] 2> "node_name":"127.0.0.1:12249_",
[junit4] 2> "roles":null,
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "shard":"shard1",
[junit4] 2> "numShards":"1",
[junit4] 2> "collection":"control_collection",
[junit4] 2> "state":"active"}
[junit4] 2> 1121559 T3994 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1121559 T4001 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1122985 T3965 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 1
[junit4] 2> 1122985 T3965 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 1123015 T3965 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
[junit4] 2> 1123017 T3965 oejs.AbstractConnector.doStart Started SslSocketConnector@127.0.0.1:12261
[junit4] 2> 1123019 T3965 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 1123020 T3965 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 1123020 T3965 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.ChaosMonkeySafeLeaderTest-D1E9884D0A34B708-001/tempDir-003
[junit4] 2> 1123020 T3965 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.ChaosMonkeySafeLeaderTest-D1E9884D0A34B708-001/tempDir-003/'
[junit4] 2> 1123056 T3965 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J3/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.ChaosMonkeySafeLeaderTest-D1E9884D0A34B708-001/tempDir-003/solr.xml
[junit4] 2> 1123123 T3965 oasc.CoreContainer.<init> New CoreContainer 1398625024
[junit4] 2> 1123123 T3965 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.ChaosMonkeySafeLeaderTest-D1E9884D0A34B708-001/tempDir-003/]
[junit4] 2> 1123124 T3965 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 1123124 T3965 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 1123124 T3965 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 1123125 T3965 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 1123125 T3965 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 1123125 T3965 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 1123125 T3965 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 1123125 T3965 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 1123126 T3965 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 1123128 T3965 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
[junit4] 2> 1123129 T3965 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 1123130 T3965 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 1123130 T3965 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 1123130 T3965 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:12240/solr
[junit4] 2> 1123130 T3965 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 1123130 T3965 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
[junit4] 2> 1123131 T3965 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1123132 T4012 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1b231b56 name:ZooKeeperConnection Watcher:127.0.0.1:12240 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1123135 T3965 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1123135 T3965 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
[junit4] 2> 1123138 T3965 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1123138 T4014 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4edf85e9 name:ZooKeeperConnection Watcher:127.0.0.1:12240/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1123139 T3965 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1123142 T3965 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1124145 T3965 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:12261_
[junit4] 2> 1124146 T3965 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:12261_
[junit4] 2> 1124149 T3965 oasc.Overseer.close Overseer (id=null) closing
[junit4] 2> 1124155 T4016 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
[junit4] 2> 1124155 T4016 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 1124156 T4016 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 1124156 T3994 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1124158 T3990 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "core":"collection1",
[junit4] 2> "base_url":"https://127.0.0.1:12261",
[junit4] 2> "node_name":"127.0.0.1:12261_",
[junit4] 2> "roles":null,
[junit4] 2> "shard":null,
[junit4] 2> "numShards":"1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "state":"down"}
[junit4] 2> 1124158 T3990 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1]
[junit4] 2> 1124158 T3990 oasc.Overseer$ClusterStateUpdater.createCollection state version collection1 1
[junit4] 2> 1124159 T3990 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 1124161 T4001 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 1124161 T3994 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 1124161 T4015 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 1125156 T4016 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 1125157 T4016 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 1125157 T4016 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 1125158 T4016 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 1125158 T4016 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 1125158 T4016 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.ChaosMonkeySafeLeaderTest-D1E9884D0A34B708-001/tempDir-003/collection1/'
[junit4] 2> 1125161 T4016 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.ChaosMonkeySafeLeaderTest-D1E9884D0A34B708-001/tempDir-003/collection1/lib/classes/' to classloader
[junit4] 2> 1125161 T4016 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.ChaosMonkeySafeLeaderTest-D1E9884D0A34B708-001/tempDir-003/collection1/lib/.svn/' to classloader
[junit4] 2> 1125162 T4016 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.ChaosMonkeySafeLeaderTest-D1E9884D0A34B708-001/tempDir-003/collection1/lib/README' to classloader
[junit4] 2> 1125224 T4016 oasc.SolrConfig.<init> Using Lucene MatchVersion: 5.0.0
[junit4] 2> 1125270 T4016 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 1125371 T4016 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 1125380 T4016 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 1125710 T4016 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 1125713 T4016 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 1125715 T4016 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 1125719 T4016 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 1125735 T4016 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection collection1
[junit4] 2> 1125736 T4016 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4] 2> 1125736 T4016 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.ChaosMonkeySafeLeaderTest-D1E9884D0A34B708-001/tempDir-003/collection1/, dataDir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.ChaosMonkeySafeLeaderTest-D1E9884D0A34B708-001/tempDir-001/jetty1/
[junit4] 2> 1125736 T4016 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@66bf378a
[junit4] 2> 1125738 T4016 oasc.CachingDirectoryFactory.get return new directory for /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.ChaosMonkeySafeLeaderTest-D1E9884D0A34B708-001/tempDir-001/jetty1
[junit4] 2> 1125739 T4016 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.ChaosMonkeySafeLeaderTest-D1E9884D0A34B708-001/tempDir-001/jetty1/index/
[junit4] 2> 1125739 T4016 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.ChaosMonkeySafeLeaderTest-D1E9884D0A34B708-001/tempDir-001/jetty1/index' doesn't exist. Creating new index...
[junit4] 2> 1125741 T4016 oasc.CachingDirectoryFactory.get return new directory for /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.ChaosMonkeySafeLeaderTest-D1E9884D0A34B708-001/tempDir-001/jetty1/index
[junit4] 2> 1125742 T4016 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=31, maxMergeAtOnceExplicit=22, maxMergedSegmentMB=93.6494140625, floorSegmentMB=0.2197265625, forceMergeDeletesPctAllowed=13.12195925824124, segmentsPerTier=23.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> 1125750 T4016 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.ChaosMonkeySafeLeaderTest-D1E9884D0A34B708-001/tempDir-001/jetty1/index,segFN=segments_1,generation=1}
[junit4] 2> 1125751 T4016 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 1125757 T4016 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 1125757 T4016 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 1125757 T4016 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 1125758 T4016 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 1125758 T4016 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 1125758 T4016 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 1125759 T4016 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 1125759 T4016 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 1125759 T4016 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 1125760 T4016 oasc.RequestHandlers.initHandlersFromConfig created /update/json: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 1125760 T4016 oasc.RequestHandlers.initHandlersFromConfig created /update/csv: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 1125761 T4016 oasc.RequestHandlers.initHandlersFromConfig created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 1125762 T4016 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 1125762 T4016 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 1125762 T4016 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 1125763 T4016 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 1125763 T4016 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 1125764 T4016 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 1125764 T4016 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 1125765 T4016 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
[junit4] 2> 1125783 T4016 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 1125785 T4016 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 1125787 T4016 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 1125789 T4016 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 1125792 T4016 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 1125792 T4016 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 1125793 T4016 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=13, maxMergeAtOnceExplicit=27, maxMergedSegmentMB=68.30859375, floorSegmentMB=1.7421875, forceMergeDeletesPctAllowed=13.621698894646508, segmentsPerTier=28.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> 1125795 T4016 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.ChaosMonkeySafeLeaderTest-D1E9884D0A34B708-001/tempDir-001/jetty1/index,segFN=segments_1,generation=1}
[junit4] 2> 1125795 T4016 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 1125795 T4016 oass.SolrIndexSearcher.<init> Opening Searcher@6f617cfc[collection1] main
[junit4] 2> 1125795 T4016 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 1125796 T4016 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 1125796 T4016 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
[junit4] 2> 1125797 T4016 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 1125797 T4016 oasr.RestManager.init Initializing RestManager with initArgs: {}
[junit4] 2> 1125797 T4016 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1125798 T4016 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream Read 38 bytes from znode /configs/conf1/_rest_managed.json
[junit4] 2> 1125798 T4016 oasr.ManagedResourceStorage.load Loaded LinkedHashMap at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1125799 T4016 oasr.ManagedResource.reloadFromStorage Loaded initArgs {} for /rest/managed
[junit4] 2> 1125799 T4016 oasr.RestManager.init Initializing 0 registered ManagedResources
[junit4] 2> 1125803 T4017 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6f617cfc[collection1] main{UninvertingDirectoryReader()}
[junit4] 2> 1125807 T4016 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 1125807 T4020 oasc.ZkController.register Register replica - core:collection1 address:https://127.0.0.1:12261 collection:collection1 shard:shard1
[junit4] 2> 1125808 T3965 oass.SolrDispatchFilter.init user.dir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J3
[junit4] 2> 1125808 T3965 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 1125808 T4020 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 1125812 T4020 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 1125813 T3965 oasc.AbstractFullDistribZkTestBase.enableAutoSoftCommit Turning on auto soft commit: 1000
[junit4] 2> 1125814 T3994 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1125814 T4020 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 1125814 T4020 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C385 name=collection1 org.apache.solr.core.SolrCore@18228199 url=https://127.0.0.1:12261/collection1 node=127.0.0.1:12261_ C385_STATE=coll:collection1 core:collection1 props:{core=collection1, base_url=https://127.0.0.1:12261, node_name=127.0.0.1:12261_, state=down}
[junit4] 2> 1125815 T4020 C385 P12261 oasc.SyncStrategy.sync Sync replicas to https://127.0.0.1:12261/collection1/
[junit4] 2> 1125815 T4020 C385 P12261 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 1125815 T4020 C385 P12261 oasc.SyncStrategy.syncToMe https://127.0.0.1:12261/collection1/ has no replicas
[junit4] 2> 1125815 T4020 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: https://127.0.0.1:12261/collection1/ shard1
[junit4] 2> 1125816 T4020 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 1125819 T4001 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 1125819 T4015 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 1125819 T3994 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 1125823 T3994 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1125824 T4001 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 1125824 T4015 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 1125824 T3994 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 1125870 T4020 oasc.ZkController.register We are https://127.0.0.1:12261/collection1/ and leader is https://127.0.0.1:12261/collection1/
[junit4] 2> 1125871 T4020 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:12261
[junit4] 2> 1125871 T4020 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 1125871 T4020 oasc.ZkController.publish publishing core=collection1 state=active collection=collection1
[junit4] 2> 1125871 T4020 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 1125872 T3994 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1125874 T3990 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "core":"collection1",
[junit4] 2> "base_url":"https://127.0.0.1:12261",
[junit4] 2> "node_name":"127.0.0.1:12261_",
[junit4] 2> "roles":null,
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "shard":"shard1",
[junit4] 2> "numShards":"1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "state":"active"}
[junit4] 2> ASYNC NEW_CORE C386 name=collection1 org.apache.solr.core.SolrCore@1c87ab4 url=https://127.0.0.1:12249/collection1 node=127.0.0.1:12249_ C386_STATE=coll:control_collection core:collection1 props:{core=collection1, base_url=https://127.0.0.1:12249, node_name=127.0.0.1:12249_, state=active, leader=true}
[junit4] 2> 1125950 T3979 C386 P12249 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1479706079288885248)} 0 5
[junit4] 2> 1125951 T3965 oascsi.CloudSolrServer.requestWithRetryOnStaleState ERROR Request to collection collection1 failed due to (510) org.apache.solr.common.SolrException: Not enough nodes to handle the request, retry? 0
[junit4] 2> 1125954 T3965 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 12249
[junit4] 2> 1125954 T3965 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=175473542
[junit4] 2> 1125954 T3965 oasc.ZkController.publish publishing core=collection1 state=down collection=control_collection
[junit4] 2> 1125955 T3965 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 1125956 T3994 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1125956 T3965 oasc.SolrCore.close [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@1c87ab4
[junit4] 2> 1125956 T3990 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4] 2> "operation":"state",
[junit4] 2> "core":"collection1",
[junit4] 2> "base_url":"https://127.0.0.1:12249",
[junit4] 2> "node_name":"127.0.0.1:12249_",
[junit4] 2> "roles":null,
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"control_collection",
[junit4] 2> "state":"down"}
[junit4] 2> 1125962 T3965 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=1,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=1,cumulative_errors=0,transaction_logs_total_size=0,transaction_logs_total_number=0}
[junit4] 2> 1125962 T3965 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4] 2> 1125962 T3965 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4] 2> 1125963 T3965 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4] 2> 1125963 T3965 C386 P12249 oasu.DirectUpdateHandler2.closeWriter Committing on IndexWriter close.
[junit4] 2> 1125967 T3965 C386 P12249 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4] 2> commit{dir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.ChaosMonkeySafeLeaderTest-D1E9884D0A34B708-001/tempDir-001/control/data/index,segFN=segments_1,generation=1}
[junit4] 2> commit{dir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.ChaosMonkeySafeLeaderTest-D1E9884D0A34B708-001/tempDir-001/control/data/index,segFN=segments_2,generation=2}
[junit4] 2> 1125968 T3965 C386 P12249 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4] 2> 1125970 T3965 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4] 2> 1125971 T3965 oasc.CachingDirectoryFactory.close Closing StandardDirectoryFactory - 2 directories currently being tracked
[junit4] 2> 1125971 T3965 oasc.CachingDirectoryFactory.closeCacheValue looking to close /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.ChaosMonkeySafeLeaderTest-D1E9884D0A34B708-001/tempDir-001/control/data [CachedDir<<refCount=0;path=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.ChaosMonkeySafeLeaderTest-D1E9884D0A34B708-001/tempDir-001/control/data;done=false>>]
[junit4] 2> 1125971 T3965 oasc.CachingDirectoryFactory.close Closing directory: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.ChaosMonkeySafeLeaderTest-D1E9884D0A34B708-001/tempDir-001/control/data
[junit4] 2> 1125972 T3965 oasc.CachingDirectoryFactory.closeCacheValue looking to close /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.ChaosMonkeySafeLeaderTest-D1E9884D0A34B708-001/tempDir-001/control/data/index [CachedDir<<refCount=0;path=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.ChaosMonkeySafeLeaderTest-D1E9884D0A34B708-001/tempDir-001/control/data/index;done=false>>]
[junit4] 2> 1125972 T3965 oasc.CachingDirectoryFactory.close Closing directory: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.ChaosMonkeySafeLeaderTest-D1E9884D0A34B708-001/tempDir-001/control/data/index
[junit4] 2> 1125973 T3965 oasc.Overseer.close Overseer (id=92481629410689028-127.0.0.1:12249_-n_0000000000) closing
[junit4] 2> 1125973 T3990 oasc.Overseer$ClusterStateUpdater.run Overseer Loop exiting : 127.0.0.1:12249_
[junit4] 2> 1125977 T4023 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:12261_
[junit4] 2> 1125977 T4023 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 1125979 T4023 oasc.Overseer.start Overseer (id=92481629410689031-127.0.0.1:12261_-n_0000000001) starting
[junit4] 2> 1125985 T4023 oasc.OverseerAutoReplicaFailoverThread.<init> Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
[junit4] 2> 1125985 T4025 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 1125987 T4024 oasc.Overseer$ClusterStateUpdater.run Replaying operations from work queue.
[junit4] 2> 1125987 T4024 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "core":"collection1",
[junit4] 2> "base_url":"https://127.0.0.1:12261",
[junit4] 2> "node_name":"127.0.0.1:12261_",
[junit4] 2> "roles":null,
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "shard":"shard1",
[junit4] 2> "numShards":"1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "state":"active"}
[junit4] 2> 1125988 T4023 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1125990 T4024 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4] 2> "operation":"state",
[junit4] 2> "core":"collection1",
[junit4] 2> "base_url":"https://127.0.0.1:12249",
[junit4] 2> "node_name":"127.0.0.1:12249_",
[junit4] 2> "roles":null,
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"control_collection",
[junit4] 2> "state":"down"}
[junit4] 2> 1125991 T4024 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 1127476 T4022 oasc.Overseer$ClusterStateUpdater.checkIfIamStillLeader ERROR could not read the data org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:347)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:344)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:61)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:344)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:383)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:126)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater$1.run(Overseer.java:333)
[junit4] 2>
[junit4] 2> 1127476 T3994 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> 1127480 T3965 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/,null}
[junit4] 2> 1130954 T3965 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 12261
[junit4] 2> 1130954 T3965 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=1398625024
[junit4] 2> 1130954 T3965 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
[junit4] 2> 1130955 T3965 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 1130956 T4023 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1130957 T3965 oasc.SolrCore.close [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@18228199
[junit4] 2> 1130959 T4024 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4] 2> "operation":"state",
[junit4] 2> "core":"collection1",
[junit4] 2> "base_url":"https://127.0.0.1:12261",
[junit4] 2> "node_name":"127.0.0.1:12261_",
[junit4] 2> "roles":null,
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "state":"down"}
[junit4] 2> 1130961 T4023 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1130967 T3965 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommit maxTime=1000ms,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0,transaction_logs_total_size=0,transaction_logs_total_number=0}
[junit4] 2> 1130968 T3965 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4] 2> 1130968 T3965 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4] 2> 1130968 T3965 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4] 2> 1130971 T3965 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4] 2> 1130972 T3965 oasc.CachingDirectoryFactory.close Closing StandardDirectoryFactory - 2 directories currently being tracked
[junit4] 2> 1130972 T3965 oasc.CachingDirectoryFactory.closeCacheValue looking to close /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.ChaosMonkeySafeLeaderTest-D1E9884D0A34B708-001/tempDir-001/jetty1 [CachedDir<<refCount=0;path=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.ChaosMonkeySafeLeaderTest-D1E9884D0A34B708-001/tempDir-001/jetty1;done=false>>]
[junit4] 2> 1130972 T3965 oasc.CachingDirectoryFactory.close Closing directory: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.ChaosMonkeySafeLeaderTest-D1E9884D0A34B708-001/tempDir-001/jetty1
[junit4] 2> 1130973 T3965 oasc.CachingDirectoryFactory.closeCacheValue looking to close /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.ChaosMonkeySafeLeaderTest-D1E9884D0A34B708-001/tempDir-001/jetty1/index [CachedDir<<refCount=0;path=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.ChaosMonkeySafeLeaderTest-D1E9884D0A34B708-001/tempDir-001/jetty1/index;done=false>>]
[junit4] 2> 1130973 T3965 oasc.CachingDirectoryFactory.close Closing directory: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.ChaosMonkeySafeLeaderTest-D1E9884D0A34B708-001/tempDir-001/jetty1/index
[junit4] 2> 1130974 T3965 oasc.Overseer.close Overseer (id=92481629410689031-127.0.0.1:12261_-n_0000000001) closing
[junit4] 2> 1130974 T4024 oasc.Overseer$ClusterStateUpdater.run Overseer Loop exiting : 127.0.0.1:12261_
[junit4] 2> 1132476 T4027 oasc.Overseer$ClusterStateUpdater.checkIfIamStillLeader ERROR could not read the data org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:347)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:344)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:61)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:344)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:383)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:126)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater$1.run(Overseer.java:333)
[junit4] 2>
[junit4] 2> 1132476 T4023 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> 1132479 T3965 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/,null}
[junit4] 2> 1133016 T3965 oas.SolrTestCaseJ4.tearDown ###Ending testDistribSearch
[junit4] 2> 1133018 T3965 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:12240 12240
[junit4] 2> 1133110 T3966 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:12240 12240
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=ChaosMonkeySafeLeaderTest -Dtests.method=testDistribSearch -Dtests.seed=D1E9884D0A34B708 -Dtests.multiplier=2 -Dtests.slow=true -Dtests.locale=ar_MA -Dtests.timezone=Africa/Bangui -Dtests.file.encoding=UTF-8
[junit4] ERROR 15.5s J3 | ChaosMonkeySafeLeaderTest.testDistribSearch <<<
[junit4] > Throwable #1: java.lang.NullPointerException
[junit4] > at __randomizedtesting.SeedInfo.seed([D1E9884D0A34B708:500F06557D6BD734]:0)
[junit4] > at org.apache.solr.client.solrj.impl.CloudSolrServer.requestWithRetryOnStaleState(CloudSolrServer.java:706)
[junit4] > at org.apache.solr.client.solrj.impl.CloudSolrServer.request(CloudSolrServer.java:601)
[junit4] > at org.apache.solr.client.solrj.request.AbstractUpdateRequest.process(AbstractUpdateRequest.java:124)
[junit4] > at org.apache.solr.client.solrj.SolrServer.deleteByQuery(SolrServer.java:285)
[junit4] > at org.apache.solr.client.solrj.SolrServer.deleteByQuery(SolrServer.java:271)
[junit4] > at org.apache.solr.cloud.AbstractFullDistribZkTestBase.del(AbstractFullDistribZkTestBase.java:729)
[junit4] > at org.apache.solr.cloud.ChaosMonkeySafeLeaderTest.tryDelete(ChaosMonkeySafeLeaderTest.java:194)
[junit4] > at org.apache.solr.cloud.ChaosMonkeySafeLeaderTest.doTest(ChaosMonkeySafeLeaderTest.java:112)
[junit4] > at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:869)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 1133127 T3965 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4] 2> NOTE: leaving temporary files on disk at: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.ChaosMonkeySafeLeaderTest-D1E9884D0A34B708-001
[junit4] 2> 15509 T3964 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=Lucene410: {}, docValues:{}, sim=DefaultSimilarity, locale=ar_MA, timezone=Africa/Bangui
[junit4] 2> NOTE: FreeBSD 9.1-RELEASE-p3 amd64/Oracle Corporation 1.7.0_65 (64-bit)/cpus=16,threads=1,free=152654672,total=361234432
[junit4] 2> NOTE: All tests run in this JVM: [RequestHandlersTest, TestSolrIndexConfig, DeleteReplicaTest, JSONWriterTest, PeerSyncTest, BlockCacheTest, BadComponentTest, TestGroupingSearch, SuggesterTSTTest, QueryEqualityTest, TestImplicitCoreProperties, TestRawResponseWriter, PluginInfoTest, TestConfigSets, PrimitiveFieldTypeTest, MultiTermTest, ShardRoutingTest, TestCoreContainer, TestIBSimilarityFactory, SoftAutoCommitTest, TestPhraseSuggestions, QueryResultKeyTest, DistribDocExpirationUpdateProcessorTest, AlternateDirectoryTest, IndexSchemaRuntimeFieldTest, ChangedSchemaMergeTest, NoCacheHeaderTest, TestDistributedGrouping, WordBreakSolrSpellCheckerTest, NumericFieldsTest, HdfsChaosMonkeySafeLeaderTest, TestFaceting, TestFuzzyAnalyzedSuggestions, ConnectionManagerTest, SolrIndexSplitterTest, HdfsRecoveryZkTest, SpellCheckComponentTest, TestSolrQueryParser, TestFieldCollectionResource, TestInitQParser, DistributedSuggestComponentTest, SampleTest, TestXIncludeConfig, SpellCheckCollatorTest, TestSearchPerf, DocumentAnalysisRequestHandlerTest, HighlighterTest, IndexBasedSpellCheckerTest, CachingDirectoryFactoryTest, TestSchemaVersionResource, TestPerFieldSimilarity, TestZkChroot, SolrPluginUtilsTest, SyncSliceTest, ShardSplitTest, ChaosMonkeySafeLeaderTest]
[junit4] Completed on J3 in 15.78s, 1 test, 1 error <<< FAILURES!
[...truncated 702 lines...]
BUILD FAILED
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/build.xml:491: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/build.xml:471: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/build.xml:60: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/extra-targets.xml:39: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build.xml:189: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/common-build.xml:496: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/lucene/common-build.xml:1359: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/lucene/common-build.xml:962: There were test failures: 429 suites, 1772 tests, 1 error, 59 ignored (24 assumptions)
Total time: 50 minutes 46 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Sending artifact delta relative to Lucene-Solr-Tests-5.x-Java7 #2117
Archived 5 artifacts
Archive block size is 32768
Received 0 blocks and 180436369 bytes
Compression is 0.0%
Took 33 sec
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure