You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2014/12/24 03:06:42 UTC

[JENKINS] Lucene-Solr-Tests-5.x-Java7 - Build # 2371 - Still Failing

Build: https://builds.apache.org/job/Lucene-Solr-Tests-5.x-Java7/2371/

All tests passed

Build Log:
[...truncated 53085 lines...]
-ecj-javadoc-lint-src:
 [ecj-lint] Compiling 687 source files
 [ecj-lint] invalid Class-Path header in manifest of jar file: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/core/lib/org.restlet-2.1.1.jar
 [ecj-lint] invalid Class-Path header in manifest of jar file: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/core/lib/org.restlet.ext.servlet-2.1.1.jar
 [ecj-lint] ----------
 [ecj-lint] 1. WARNING in /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/core/src/java/org/apache/solr/analysis/LegacyHTMLStripCharFilter.java (at line 72)
 [ecj-lint] 	Reader in = new LegacyHTMLStripCharFilter(
 [ecj-lint] 	       ^^
 [ecj-lint] Resource leak: 'in' is never closed
 [ecj-lint] ----------
 [ecj-lint] ----------
 [ecj-lint] 2. WARNING in /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/core/src/java/org/apache/solr/core/SolrResourceLoader.java (at line 843)
 [ecj-lint] 	final FileOutputStream out = new FileOutputStream(managedSchemaFile);
 [ecj-lint] 	                       ^^^
 [ecj-lint] Resource leak: 'out' is never closed
 [ecj-lint] ----------
 [ecj-lint] ----------
 [ecj-lint] 3. WARNING in /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/core/src/java/org/apache/solr/handler/AnalysisRequestHandlerBase.java (at line 111)
 [ecj-lint] 	reader = cfiltfac.create(reader);
 [ecj-lint] 	^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 [ecj-lint] Resource leak: 'reader' is not closed at this location
 [ecj-lint] ----------
 [ecj-lint] 4. WARNING in /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/core/src/java/org/apache/solr/handler/AnalysisRequestHandlerBase.java (at line 138)
 [ecj-lint] 	return namedList;
 [ecj-lint] 	^^^^^^^^^^^^^^^^^
 [ecj-lint] Resource leak: 'listBasedTokenStream' is not closed at this location
 [ecj-lint] ----------
 [ecj-lint] ----------
 [ecj-lint] 5. WARNING in /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/core/src/java/org/apache/solr/handler/ReplicationHandler.java (at line 941)
 [ecj-lint] 	DirectoryReader reader = s==null ? null : s.get().getIndexReader();
 [ecj-lint] 	                ^^^^^^
 [ecj-lint] Resource leak: 'reader' is never closed
 [ecj-lint] ----------
 [ecj-lint] ----------
 [ecj-lint] 6. ERROR in /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/core/src/java/org/apache/solr/handler/RequestHandlerBase.java (at line 240)
 [ecj-lint] 	* @param pluginInfo
 [ecj-lint] 	         ^^^^^^^^^^
 [ecj-lint] Javadoc: Description expected after this reference
 [ecj-lint] ----------
 [ecj-lint] ----------
 [ecj-lint] 7. WARNING in /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/core/src/java/org/apache/solr/schema/OpenExchangeRatesOrgProvider.java (at line 140)
 [ecj-lint] 	ratesJsonStream = resourceLoader.openResource(ratesFileLocation);
 [ecj-lint] 	^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 [ecj-lint] Resource leak: 'ratesJsonStream' is not closed at this location
 [ecj-lint] ----------
 [ecj-lint] ----------
 [ecj-lint] 8. WARNING in /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/core/src/java/org/apache/solr/search/function/OrdFieldSource.java (at line 79)
 [ecj-lint] 	SolrIndexSearcher is = (SolrIndexSearcher) o;
 [ecj-lint] 	                  ^^
 [ecj-lint] Resource leak: 'is' is never closed
 [ecj-lint] ----------
 [ecj-lint] ----------
 [ecj-lint] 9. WARNING in /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/core/src/java/org/apache/solr/search/function/ReverseOrdFieldSource.java (at line 79)
 [ecj-lint] 	SolrIndexSearcher is = (SolrIndexSearcher) o;
 [ecj-lint] 	                  ^^
 [ecj-lint] Resource leak: 'is' is never closed
 [ecj-lint] ----------
 [ecj-lint] ----------
 [ecj-lint] 10. WARNING in /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/core/src/java/org/apache/solr/update/TransactionLog.java (at line 218)
 [ecj-lint] 	ChannelFastInputStream is = new ChannelFastInputStream(channel, pos);
 [ecj-lint] 	                       ^^
 [ecj-lint] Resource leak: 'is' is never closed
 [ecj-lint] ----------
 [ecj-lint] 11. WARNING in /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/core/src/java/org/apache/solr/update/TransactionLog.java (at line 218)
 [ecj-lint] 	ChannelFastInputStream is = new ChannelFastInputStream(channel, pos);
 [ecj-lint] 	                       ^^
 [ecj-lint] Resource leak: 'is' is never closed
 [ecj-lint] ----------
 [ecj-lint] ----------
 [ecj-lint] 12. WARNING in /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/core/src/java/org/apache/solr/util/SimplePostTool.java (at line 640)
 [ecj-lint] 	BAOS bos =  new BAOS();
 [ecj-lint] 	     ^^^
 [ecj-lint] Resource leak: 'bos' is never closed
 [ecj-lint] ----------
 [ecj-lint] 13. WARNING in /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/core/src/java/org/apache/solr/util/SimplePostTool.java (at line 640)
 [ecj-lint] 	BAOS bos =  new BAOS();
 [ecj-lint] 	     ^^^
 [ecj-lint] Resource leak: 'bos' is never closed
 [ecj-lint] ----------
 [ecj-lint] 13 problems (1 error, 12 warnings)

BUILD FAILED
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/build.xml:529: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/build.xml:83: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build.xml:580: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/lucene/common-build.xml:1913: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/lucene/common-build.xml:1947: Compile failed; see the compiler error output for details.

Total time: 102 minutes 35 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Sending artifact delta relative to Lucene-Solr-Tests-5.x-Java7 #2368
Archived 1 artifacts
Archive block size is 32768
Received 0 blocks and 464 bytes
Compression is 0.0%
Took 0.45 sec
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-Tests-5.x-Java7 - Build # 2373 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-5.x-Java7/2373/

All tests passed

Build Log:
[...truncated 53368 lines...]
-ecj-javadoc-lint-src:
 [ecj-lint] Compiling 687 source files
 [ecj-lint] invalid Class-Path header in manifest of jar file: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/core/lib/org.restlet-2.1.1.jar
 [ecj-lint] invalid Class-Path header in manifest of jar file: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/core/lib/org.restlet.ext.servlet-2.1.1.jar
 [ecj-lint] ----------
 [ecj-lint] 1. WARNING in /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/core/src/java/org/apache/solr/analysis/LegacyHTMLStripCharFilter.java (at line 72)
 [ecj-lint] 	Reader in = new LegacyHTMLStripCharFilter(
 [ecj-lint] 	       ^^
 [ecj-lint] Resource leak: 'in' is never closed
 [ecj-lint] ----------
 [ecj-lint] ----------
 [ecj-lint] 2. WARNING in /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/core/src/java/org/apache/solr/core/SolrResourceLoader.java (at line 843)
 [ecj-lint] 	final FileOutputStream out = new FileOutputStream(managedSchemaFile);
 [ecj-lint] 	                       ^^^
 [ecj-lint] Resource leak: 'out' is never closed
 [ecj-lint] ----------
 [ecj-lint] ----------
 [ecj-lint] 3. WARNING in /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/core/src/java/org/apache/solr/handler/AnalysisRequestHandlerBase.java (at line 111)
 [ecj-lint] 	reader = cfiltfac.create(reader);
 [ecj-lint] 	^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 [ecj-lint] Resource leak: 'reader' is not closed at this location
 [ecj-lint] ----------
 [ecj-lint] 4. WARNING in /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/core/src/java/org/apache/solr/handler/AnalysisRequestHandlerBase.java (at line 138)
 [ecj-lint] 	return namedList;
 [ecj-lint] 	^^^^^^^^^^^^^^^^^
 [ecj-lint] Resource leak: 'listBasedTokenStream' is not closed at this location
 [ecj-lint] ----------
 [ecj-lint] ----------
 [ecj-lint] 5. WARNING in /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/core/src/java/org/apache/solr/handler/ReplicationHandler.java (at line 941)
 [ecj-lint] 	DirectoryReader reader = s==null ? null : s.get().getIndexReader();
 [ecj-lint] 	                ^^^^^^
 [ecj-lint] Resource leak: 'reader' is never closed
 [ecj-lint] ----------
 [ecj-lint] ----------
 [ecj-lint] 6. ERROR in /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/core/src/java/org/apache/solr/handler/RequestHandlerBase.java (at line 240)
 [ecj-lint] 	* @param pluginInfo
 [ecj-lint] 	         ^^^^^^^^^^
 [ecj-lint] Javadoc: Description expected after this reference
 [ecj-lint] ----------
 [ecj-lint] ----------
 [ecj-lint] 7. WARNING in /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/core/src/java/org/apache/solr/schema/OpenExchangeRatesOrgProvider.java (at line 140)
 [ecj-lint] 	ratesJsonStream = resourceLoader.openResource(ratesFileLocation);
 [ecj-lint] 	^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 [ecj-lint] Resource leak: 'ratesJsonStream' is not closed at this location
 [ecj-lint] ----------
 [ecj-lint] ----------
 [ecj-lint] 8. WARNING in /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/core/src/java/org/apache/solr/search/function/OrdFieldSource.java (at line 79)
 [ecj-lint] 	SolrIndexSearcher is = (SolrIndexSearcher) o;
 [ecj-lint] 	                  ^^
 [ecj-lint] Resource leak: 'is' is never closed
 [ecj-lint] ----------
 [ecj-lint] ----------
 [ecj-lint] 9. WARNING in /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/core/src/java/org/apache/solr/search/function/ReverseOrdFieldSource.java (at line 79)
 [ecj-lint] 	SolrIndexSearcher is = (SolrIndexSearcher) o;
 [ecj-lint] 	                  ^^
 [ecj-lint] Resource leak: 'is' is never closed
 [ecj-lint] ----------
 [ecj-lint] ----------
 [ecj-lint] 10. WARNING in /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/core/src/java/org/apache/solr/update/TransactionLog.java (at line 218)
 [ecj-lint] 	ChannelFastInputStream is = new ChannelFastInputStream(channel, pos);
 [ecj-lint] 	                       ^^
 [ecj-lint] Resource leak: 'is' is never closed
 [ecj-lint] ----------
 [ecj-lint] 11. WARNING in /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/core/src/java/org/apache/solr/update/TransactionLog.java (at line 218)
 [ecj-lint] 	ChannelFastInputStream is = new ChannelFastInputStream(channel, pos);
 [ecj-lint] 	                       ^^
 [ecj-lint] Resource leak: 'is' is never closed
 [ecj-lint] ----------
 [ecj-lint] ----------
 [ecj-lint] 12. WARNING in /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/core/src/java/org/apache/solr/util/SimplePostTool.java (at line 640)
 [ecj-lint] 	BAOS bos =  new BAOS();
 [ecj-lint] 	     ^^^
 [ecj-lint] Resource leak: 'bos' is never closed
 [ecj-lint] ----------
 [ecj-lint] 13. WARNING in /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/core/src/java/org/apache/solr/util/SimplePostTool.java (at line 640)
 [ecj-lint] 	BAOS bos =  new BAOS();
 [ecj-lint] 	     ^^^
 [ecj-lint] Resource leak: 'bos' is never closed
 [ecj-lint] ----------
 [ecj-lint] 13 problems (1 error, 12 warnings)

BUILD FAILED
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/build.xml:529: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/build.xml:83: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build.xml:580: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/lucene/common-build.xml:1913: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/lucene/common-build.xml:1947: Compile failed; see the compiler error output for details.

Total time: 100 minutes 17 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Sending artifact delta relative to Lucene-Solr-Tests-5.x-Java7 #2368
Archived 1 artifacts
Archive block size is 32768
Received 0 blocks and 464 bytes
Compression is 0.0%
Took 0.18 sec
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-Tests-5.x-Java7 - Build # 2372 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-5.x-Java7/2372/

2 tests failed.
REGRESSION:  org.apache.solr.cloud.ChaosMonkeySafeLeaderTest.testDistribSearch

Error Message:
Test abandoned because suite timeout was reached.

Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
	at __randomizedtesting.SeedInfo.seed([A7D30E6BB32BE25E]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.ChaosMonkeySafeLeaderTest

Error Message:
Suite timeout exceeded (>= 7200000 msec).

Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
	at __randomizedtesting.SeedInfo.seed([A7D30E6BB32BE25E]:0)




Build Log:
[...truncated 10362 lines...]
   [junit4] Suite: org.apache.solr.cloud.ChaosMonkeySafeLeaderTest
   [junit4]   2> Creating dataDir: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.cloud.ChaosMonkeySafeLeaderTest-A7D30E6BB32BE25E-001/init-core-data-001
   [junit4]   2> 1004543 T1728 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (true) and clientAuth (false)
   [junit4]   2> 1004543 T1728 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
   [junit4]   2> 1004551 T1728 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
   [junit4]   2> 1004552 T1728 oasc.ZkTestServer.run STARTING ZK TEST SERVER
   [junit4]   1> client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1004553 T1729 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
   [junit4]   2> 1004653 T1728 oasc.ZkTestServer.run start zk server on port:38889
   [junit4]   2> 1004654 T1728 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 1004655 T1728 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1004660 T1736 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5288eedc name:ZooKeeperConnection Watcher:127.0.0.1:38889 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1004660 T1728 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1004661 T1728 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 1004661 T1728 oascc.SolrZkClient.makePath makePath: /solr
   [junit4]   2> 1004664 T1728 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 1004665 T1728 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1004667 T1739 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@60f06026 name:ZooKeeperConnection Watcher:127.0.0.1:38889/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1004667 T1728 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1004667 T1728 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 1004668 T1728 oascc.SolrZkClient.makePath makePath: /collections/collection1
   [junit4]   2> 1004670 T1728 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
   [junit4]   2> 1004671 T1728 oascc.SolrZkClient.makePath makePath: /collections/control_collection
   [junit4]   2> 1004673 T1728 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
   [junit4]   2> 1004675 T1728 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 1004675 T1728 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 1004678 T1728 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/core/src/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
   [junit4]   2> 1004679 T1728 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
   [junit4]   2> 1004782 T1728 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1004783 T1728 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1004784 T1728 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 1004785 T1728 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 1004787 T1728 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 1004787 T1728 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
   [junit4]   2> 1004789 T1728 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 1004789 T1728 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
   [junit4]   2> 1004791 T1728 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 1004792 T1728 oascc.SolrZkClient.makePath makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 1004794 T1728 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1004794 T1728 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1004796 T1728 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1004797 T1728 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1004799 T1728 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 1004800 T1728 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 1004801 T1728 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 1004802 T1728 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 1004804 T1728 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 1004805 T1728 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1004807 T1742 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1e87130a name:ZooKeeperConnection Watcher:127.0.0.1:38889/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1004807 T1728 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1004807 T1728 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 1006020 T1728 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 1006089 T1728 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
   [junit4]   2> 1006094 T1728 oejs.AbstractConnector.doStart Started SslSocketConnector@127.0.0.1:38820
   [junit4]   2> 1006098 T1728 oass.SolrDispatchFilter.init SolrDispatchFilter.init()sun.misc.Launcher$AppClassLoader@57bd06bf
   [junit4]   2> 1006098 T1728 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1006098 T1728 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.cloud.ChaosMonkeySafeLeaderTest-A7D30E6BB32BE25E-001/tempDir-002
   [junit4]   2> 1006099 T1728 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.cloud.ChaosMonkeySafeLeaderTest-A7D30E6BB32BE25E-001/tempDir-002/'
   [junit4]   2> 1006129 T1728 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.cloud.ChaosMonkeySafeLeaderTest-A7D30E6BB32BE25E-001/tempDir-002/solr.xml
   [junit4]   2> 1006184 T1728 oasc.CoreContainer.<init> New CoreContainer 2030147318
   [junit4]   2> 1006185 T1728 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.cloud.ChaosMonkeySafeLeaderTest-A7D30E6BB32BE25E-001/tempDir-002/]
   [junit4]   2> 1006186 T1728 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 1006186 T1728 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 1006186 T1728 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 1006186 T1728 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 1006187 T1728 oashc.HttpShardHandlerFactory.getParameter Setting maxConnections to: 10000
   [junit4]   2> 1006187 T1728 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 1006187 T1728 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 1006187 T1728 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 1006188 T1728 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 1006188 T1728 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 1006191 T1728 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
   [junit4]   2> 1006194 T1728 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1006194 T1728 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1006194 T1728 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 1006194 T1728 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:38889/solr
   [junit4]   2> 1006195 T1728 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 1006195 T1728 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 1006195 T1728 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1006198 T1754 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3262d5 name:ZooKeeperConnection Watcher:127.0.0.1:38889 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1006199 T1728 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1006199 T1728 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 1006201 T1728 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1006202 T1757 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@44e0dc96 name:ZooKeeperConnection Watcher:127.0.0.1:38889/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1006202 T1728 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1006203 T1728 oascc.SolrZkClient.makePath makePath: /overseer/queue
   [junit4]   2> 1006204 T1728 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
   [junit4]   2> 1006205 T1728 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
   [junit4]   2> 1006207 T1728 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
   [junit4]   2> 1006208 T1728 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
   [junit4]   2> 1006210 T1728 oascc.SolrZkClient.makePath makePath: /live_nodes
   [junit4]   2> 1006210 T1728 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:38820_
   [junit4]   2> 1006211 T1728 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:38820_
   [junit4]   2> 1006212 T1728 oascc.SolrZkClient.makePath makePath: /overseer_elect
   [junit4]   2> 1006213 T1728 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
   [junit4]   2> 1006214 T1728 oasc.Overseer.close Overseer (id=null) closing
   [junit4]   2> 1006215 T1728 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:38820_
   [junit4]   2> 1006215 T1728 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
   [junit4]   2> 1006216 T1728 oasc.Overseer.start Overseer (id=93021542515605508-127.0.0.1:38820_-n_0000000000) starting
   [junit4]   2> 1006217 T1728 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
   [junit4]   2> 1006221 T1728 oasc.OverseerAutoReplicaFailoverThread.<init> Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 1006222 T1759 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
   [junit4]   2> 1006222 T1728 oascc.SolrZkClient.makePath makePath: /clusterstate.json
   [junit4]   2> 1006223 T1728 oascc.SolrZkClient.makePath makePath: /aliases.json
   [junit4]   2> 1006224 T1728 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 1006226 T1758 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
   [junit4]   2> 1006230 T1761 oasc.ZkController.publish publishing core=collection1 state=down collection=control_collection
   [junit4]   2> 1006230 T1761 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 1006231 T1761 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 1006231 T1757 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 1006234 T1758 oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 0, message = {
   [junit4]   2> 	  "base_url":"https://127.0.0.1:38820",
   [junit4]   2> 	  "numShards":"1",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:38820_",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "collection":"control_collection",
   [junit4]   2> 	  "roles":null} current state version: 0
   [junit4]   2> 1006234 T1758 oasco.ReplicaMutator.updateState Update state numShards=1 message={
   [junit4]   2> 	  "base_url":"https://127.0.0.1:38820",
   [junit4]   2> 	  "numShards":"1",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:38820_",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "collection":"control_collection",
   [junit4]   2> 	  "roles":null}
   [junit4]   2> 1006235 T1758 oasco.ClusterStateMutator.createCollection building a new cName: control_collection
   [junit4]   2> 1006235 T1758 oasco.ReplicaMutator.updateState Assigning new node to shard shard=shard1
   [junit4]   2> 1006236 T1757 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1007232 T1761 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1007233 T1761 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
   [junit4]   2> 1007233 T1761 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 1007234 T1761 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
   [junit4]   2> 1007234 T1761 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1007235 T1761 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.cloud.ChaosMonkeySafeLeaderTest-A7D30E6BB32BE25E-001/tempDir-002/collection1/'
   [junit4]   2> 1007237 T1761 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.cloud.ChaosMonkeySafeLeaderTest-A7D30E6BB32BE25E-001/tempDir-002/collection1/lib/.svn/' to classloader
   [junit4]   2> 1007238 T1761 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.cloud.ChaosMonkeySafeLeaderTest-A7D30E6BB32BE25E-001/tempDir-002/collection1/lib/classes/' to classloader
   [junit4]   2> 1007238 T1761 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.cloud.ChaosMonkeySafeLeaderTest-A7D30E6BB32BE25E-001/tempDir-002/collection1/lib/README' to classloader
   [junit4]   2> 1007272 T1761 oasc.ZkController.watchZKConfDir watch zkdir /configs/conf1
   [junit4]   2> 1007274 T1761 oasc.Config.<init> loaded config solrconfig.xml with version 0 
   [junit4]   2> 1007308 T1761 oasc.SolrConfig.<init> Using Lucene MatchVersion: 5.0.0
   [junit4]   2> 1007356 T1761 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1007458 T1761 oass.IndexSchema.readSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 1007469 T1761 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 1007824 T1761 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 1007827 T1761 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 1007829 T1761 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 1007834 T1761 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 1007857 T1761 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 1007858 T1761 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
   [junit4]   2> 1007858 T1761 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.cloud.ChaosMonkeySafeLeaderTest-A7D30E6BB32BE25E-001/tempDir-002/collection1/, dataDir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.cloud.ChaosMonkeySafeLeaderTest-A7D30E6BB32BE25E-001/tempDir-001/control/data/
   [junit4]   2> 1007858 T1761 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@5089dee5
   [junit4]   2> 1007861 T1761 oasc.CachingDirectoryFactory.get return new directory for /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.cloud.ChaosMonkeySafeLeaderTest-A7D30E6BB32BE25E-001/tempDir-001/control/data
   [junit4]   2> 1007862 T1761 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.cloud.ChaosMonkeySafeLeaderTest-A7D30E6BB32BE25E-001/tempDir-001/control/data/index/
   [junit4]   2> 1007863 T1761 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.cloud.ChaosMonkeySafeLeaderTest-A7D30E6BB32BE25E-001/tempDir-001/control/data/index' doesn't exist. Creating new index...
   [junit4]   2> 1007865 T1761 oasc.CachingDirectoryFactory.get return new directory for /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.cloud.ChaosMonkeySafeLeaderTest-A7D30E6BB32BE25E-001/tempDir-001/control/data/index
   [junit4]   2> 1007865 T1761 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=48, maxMergeAtOnceExplicit=23, maxMergedSegmentMB=85.8037109375, floorSegmentMB=0.783203125, forceMergeDeletesPctAllowed=24.188309162155292, segmentsPerTier=33.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 1007870 T1761 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.cloud.ChaosMonkeySafeLeaderTest-A7D30E6BB32BE25E-001/tempDir-001/control/data/index,segFN=segments_1,generation=1}
   [junit4]   2> 1007870 T1761 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 1007876 T1761 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 1007877 T1761 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 1007877 T1761 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 1007877 T1761 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1007878 T1761 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1007878 T1761 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 1007878 T1761 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1007879 T1761 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1007879 T1761 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 1007880 T1761 oasc.RequestHandlers.initHandlersFromConfig created /update: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 1007880 T1761 oasc.RequestHandlers.initHandlersFromConfig created /update/json: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 1007881 T1761 oasc.RequestHandlers.initHandlersFromConfig created /update/csv: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 1007881 T1761 oasc.RequestHandlers.initHandlersFromConfig created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 1007882 T1761 oasc.RequestHandlers.initHandlersFromConfig created /config: org.apache.solr.handler.SolrConfigHandler
   [junit4]   2> 1007882 T1761 oasc.RequestHandlers.initHandlersFromConfig created /schema: org.apache.solr.handler.SchemaHandler
   [junit4]   2> 1007883 T1761 oasc.RequestHandlers.initHandlersFromConfig created /replication: org.apache.solr.handler.ReplicationHandler
   [junit4]   2> 1007884 T1761 oasc.RequestHandlers.initHandlersFromConfig created /get: org.apache.solr.handler.RealTimeGetHandler
   [junit4]   2> 1007884 T1761 oasc.RequestHandlers.initHandlersFromConfig created /admin/luke: org.apache.solr.handler.admin.LukeRequestHandler
   [junit4]   2> 1007886 T1761 oasc.RequestHandlers.initHandlersFromConfig created /admin/system: org.apache.solr.handler.admin.SystemInfoHandler
   [junit4]   2> 1007886 T1761 oasc.RequestHandlers.initHandlersFromConfig created /admin/mbeans: org.apache.solr.handler.admin.SolrInfoMBeanHandler
   [junit4]   2> 1007887 T1761 oasc.RequestHandlers.initHandlersFromConfig created /admin/plugins: org.apache.solr.handler.admin.PluginInfoHandler
   [junit4]   2> 1007887 T1761 oasc.RequestHandlers.initHandlersFromConfig created /admin/threads: org.apache.solr.handler.admin.ThreadDumpHandler
   [junit4]   2> 1007888 T1761 oasc.RequestHandlers.initHandlersFromConfig created /admin/properties: org.apache.solr.handler.admin.PropertiesRequestHandler
   [junit4]   2> 1007888 T1761 oasc.RequestHandlers.initHandlersFromConfig created /admin/logging: org.apache.solr.handler.admin.LoggingHandler
   [junit4]   2> 1007889 T1761 oasc.RequestHandlers.initHandlersFromConfig created /admin/ping: org.apache.solr.handler.PingRequestHandler
   [junit4]   2> 1007890 T1761 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 1007890 T1761 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
   [junit4]   2> 1007891 T1761 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
   [junit4]   2> 1007909 T1761 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 1007911 T1761 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 1007913 T1761 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 1007915 T1761 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 1007919 T1761 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 1007919 T1761 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 1007920 T1761 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=11, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.6068313772277054]
   [junit4]   2> 1007921 T1761 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.cloud.ChaosMonkeySafeLeaderTest-A7D30E6BB32BE25E-001/tempDir-001/control/data/index,segFN=segments_1,generation=1}
   [junit4]   2> 1007922 T1761 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 1007922 T1761 oass.SolrIndexSearcher.<init> Opening Searcher@1d874c31[collection1] main
   [junit4]   2> 1007922 T1761 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
   [junit4]   2> 1007923 T1761 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1007923 T1761 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 1007924 T1761 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 1007924 T1761 oasr.RestManager.init Initializing RestManager with initArgs: {}
   [junit4]   2> 1007924 T1761 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1007925 T1761 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 1007925 T1761 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1007926 T1761 oascc.SolrZkClient.makePath makePath: /configs/conf1/_rest_managed.json
   [junit4]   2> 1007928 T1761 oasr.ManagedResourceStorage$ZooKeeperStorageIO$1.close Wrote 38 bytes to new znode /configs/conf1/_rest_managed.json
   [junit4]   2> 1007928 T1761 oasr.ManagedResourceStorage$JsonStorage.store Saved JSON object to path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1007928 T1761 oasr.RestManager.init Initializing 0 registered ManagedResources
   [junit4]   2> 1007928 T1761 oash.ReplicationHandler.inform Commits will be reserved for  10000
   [junit4]   2> 1007929 T1762 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1d874c31[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1007930 T1761 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 1007931 T1765 oasc.ZkController.register Register replica - core:collection1 address:https://127.0.0.1:38820 collection:control_collection shard:shard1
   [junit4]   2> 1007931 T1728 oass.SolrDispatchFilter.init user.dir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2
   [junit4]   2> 1007932 T1728 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 1007932 T1765 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 1007936 T1765 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
   [junit4]   2> 1007938 T1757 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 1007938 T1765 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
   [junit4]   2> 1007938 T1765 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
   [junit4]   2> ASYNC  NEW_CORE C264 name=collection1 org.apache.solr.core.SolrCore@2989b558 url=https://127.0.0.1:38820/collection1 node=127.0.0.1:38820_ C264_STATE=coll:control_collection core:collection1 props:{base_url=https://127.0.0.1:38820, node_name=127.0.0.1:38820_, core=collection1, state=down}
   [junit4]   2> 1007938 T1765 C264 P38820 oasc.SyncStrategy.sync Sync replicas to https://127.0.0.1:38820/collection1/
   [junit4]   2> 1007939 T1758 oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "operation":"leader",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "collection":"control_collection"} current state version: 1
   [junit4]   2> 1007939 T1765 C264 P38820 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
   [junit4]   2> 1007939 T1765 C264 P38820 oasc.SyncStrategy.syncToMe https://127.0.0.1:38820/collection1/ has no replicas
   [junit4]   2> 1007939 T1765 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: https://127.0.0.1:38820/collection1/ shard1
   [junit4]   2> 1007940 T1765 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 1007940 T1757 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1007940 T1728 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 1007941 T1728 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1007944 T1768 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@197b26 name:ZooKeeperConnection Watcher:127.0.0.1:38889/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1007944 T1728 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1007944 T1728 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 1007946 T1769 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 1007946 T1728 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 1007946 T1758 oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "operation":"leader",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "collection":"control_collection",
   [junit4]   2> 	  "base_url":"https://127.0.0.1:38820",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "state":"active"} current state version: 2
   [junit4]   2> 1007948 T1728 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 1008050 T1768 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1008099 T1765 oasc.ZkController.register We are https://127.0.0.1:38820/collection1/ and leader is https://127.0.0.1:38820/collection1/
   [junit4]   2> 1008100 T1765 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:38820
   [junit4]   2> 1008100 T1765 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
   [junit4]   2> 1008100 T1765 oasc.ZkController.publish publishing core=collection1 state=active collection=control_collection
   [junit4]   2> 1008100 T1765 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 1008101 T1757 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 1008103 T1758 oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "base_url":"https://127.0.0.1:38820",
   [junit4]   2> 	  "numShards":"3",
   [junit4]   2> 	  "core_node_name":"core_node1",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "node_name":"127.0.0.1:38820_",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "state":"active",
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "collection":"control_collection",
   [junit4]   2> 	  "roles":null} current state version: 3
   [junit4]   2> 1008104 T1758 oasco.ReplicaMutator.updateState Update state numShards=3 message={
   [junit4]   2> 	  "base_url":"https://127.0.0.1:38820",
   [junit4]   2> 	  "numShards":"3",
   [junit4]   2> 	  "core_node_name":"core_node1",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "node_name":"127.0.0.1:38820_",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "state":"active",
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "collection":"control_collection",
   [junit4]   2> 	  "roles":null}
   [junit4]   2> 1008207 T1757 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1008207 T1768 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1009097 T1728 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 1
   [junit4]   2> 1009098 T1728 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 1009104 T1728 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
   [junit4]   2> 1009107 T1728 oejs.AbstractConnector.doStart Started SslSocketConnector@127.0.0.1:31330
   [junit4]   2> 1009111 T1728 oass.SolrDispatchFilter.init SolrDispatchFilter.init()sun.misc.Launcher$AppClassLoader@57bd06bf
   [junit4]   2> 1009111 T1728 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1009111 T1728 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.cloud.ChaosMonkeySafeLeaderTest-A7D30E6BB32BE25E-001/tempDir-003
   [junit4]   2> 1009112 T1728 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.cloud.ChaosMonkeySafeLeaderTest-A7D30E6BB32BE25E-001/tempDir-003/'
   [junit4]   2> 1009142 T1728 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.cloud.ChaosMonkeySafeLeaderTest-A7D30E6BB32BE25E-001/tempDir-003/solr.xml
   [junit4]   2> 1009196 T1728 oasc.CoreContainer.<init> New CoreContainer 1000284766
   [junit4]   2> 1009197 T1728 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.cloud.ChaosMonkeySafeLeaderTest-A7D30E6BB32BE25E-001/tempDir-003/]
   [junit4]   2> 1009198 T1728 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 1009198 T1728 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 1009198 T1728 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 1009198 T1728 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 1009199 T1728 oashc.HttpShardHandlerFactory.getParameter Setting maxConnections to: 10000
   [junit4]   2> 1009199 T1728 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 1009199 T1728 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 1009199 T1728 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 1009200 T1728 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 1009200 T1728 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 1009203 T1728 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
   [junit4]   2> 1009205 T1728 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1009205 T1728 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1009205 T1728 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 1009206 T1728 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:38889/solr
   [junit4]   2> 1009206 T1728 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 1009206 T1728 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 1009207 T1728 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1009209 T1781 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@cdcddb5 name:ZooKeeperConnection Watcher:127.0.0.1:38889 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1009210 T1728 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1009210 T1728 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 1009212 T1728 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1009213 T1784 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@57bbf44f name:ZooKeeperConnection Watcher:127.0.0.1:38889/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1009214 T1728 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1009218 T1728 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 1010222 T1728 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:31330_
   [junit4]   2> 1010223 T1728 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:31330_
   [junit4]   2> 1010226 T1728 oasc.Overseer.close Overseer (id=null) closing
   [junit4]   2> 1010234 T1785 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
   [junit4]   2> 1010234 T1785 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 1010235 T1757 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 1010235 T1785 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 1010236 T1758 oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "base_url":"https://127.0.0.1:31330",
   [junit4]   2> 	  "numShards":"3",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:31330_",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "roles":null} current state version: 4
   [junit4]   2> 1010236 T1758 oasco.ReplicaMutator.updateState Update state numShards=3 message={
   [junit4]   2> 	  "base_url":"https://127.0.0.1:31330",
   [junit4]   2> 	  "numShards":"3",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:31330_",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "roles":null}
   [junit4]   2> 1010237 T1758 oasco.ClusterStateMutator.createCollection building a new cName: collection1
   [junit4]   2> 1010237 T1758 oasco.ReplicaMutator.updateState Assigning new node to shard shard=shard3
   [junit4]   2> 1010238 T1757 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1010238 T1768 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1010238 T1784 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1011235 T1785 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1011236 T1785 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
   [junit4]   2> 1011237 T1785 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 1011237 T1785 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
   [junit4]   2> 1011237 T1785 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1011238 T1785 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.cloud.ChaosMonkeySafeLeaderTest-A7D30E6BB32BE25E-001/tempDir-003/collection1/'
   [junit4]   2> 1011240 T1785 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.cloud.ChaosMonkeySafeLeaderTest-A7D30E6BB32BE25E-001/tempDir-003/collection1/lib/.svn/' to classloader
   [junit4]   2> 1011241 T1785 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.cloud.ChaosMonkeySafeLeaderTest-A7D30E6BB32BE25E-001/tempDir-003/collection1/lib/classes/' to classloader
   [junit4]   2> 1011242 T1785 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.cloud.ChaosMonkeySafeLeaderTest-A7D30E6BB32BE25E-001/tempDir-003/collection1/lib/README' to classloader
   [junit4]   2> 1011277 T1785 oasc.ZkController.watchZKConfDir watch zkdir /configs/conf1
   [junit4]   2> 1011280 T1785 oasc.Config.<init> loaded config solrconfig.xml with version 0 
   [junit4]   2> 1011319 T1785 oasc.SolrConfig.<init> Using Lucene MatchVersion: 5.0.0
   [junit4]   2> 1011371 T1785 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1011474 T1785 oass.IndexSchema.readSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 1011485 T1785 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 1011857 T1785 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 1011860 T1785 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 1011862 T1785 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 1011867 T1785 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 1011887 T1785 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection collection1
   [junit4]   2> 1011888 T1785 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
   [junit4]   2> 1011888 T1785 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.cloud.ChaosMonkeySafeLeaderTest-A7D30E6BB32BE25E-001/tempDir-003/collection1/, dataDir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.cloud.ChaosMonkeySafeLeaderTest-A7D30E6BB32BE25E-001/tempDir-001/jetty1/
   [junit4]   2> 1011888 T1785 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@5089dee5
   [junit4]   2> 1011890 T1785 oasc.CachingDirectoryFactory.get return new directory for /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.cloud.ChaosMonkeySafeLeaderTest-A7D30E6BB32BE25E-001/tempDir-001/jetty1
   [junit4]   2> 1011891 T1785 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.cloud.ChaosMonkeySafeLeaderTest-A7D30E6BB32BE25E-001/tempDir-001/jetty1/index/
   [junit4]   2> 1011892 T1785 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.cloud.ChaosMonkeySafeLeaderTest-A7D30E6BB32BE25E-001/tempDir-001/jetty1/index' doesn't exist. Creating new index...
   [junit4]   2> 1011893 T1785 oasc.CachingDirectoryFactory.get return new directory for /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.cloud.ChaosMonkeySafeLeaderTest-A7D30E6BB32BE25E-001/tempDir-001/jetty1/index
   [junit4]   2> 1011894 T1785 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=48, maxMergeAtOnceExplicit=23, maxMergedSegmentMB=85.8037109375, floorSegmentMB=0.783203125, forceMergeDeletesPctAllowed=24.188309162155292, segmentsPerTier=33.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 1011900 T1785 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.cloud.ChaosMonkeySafeLeaderTest-A7D30E6BB32BE25E-001/tempDir-001/jetty1/index,segFN=segments_1,generation=1}
   [junit4]   2> 1011900 T1785 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 1011908 T1785 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 1011909 T1785 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 1011909 T1785 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 1011909 T1785 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1011910 T1785 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1011910 T1785 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 1011910 T1785 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1011911 T1785 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1011911 T1785 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 1011912 T1785 oasc.RequestHandlers.initHandlersFromConfig created /update: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 1011913 T1785 oasc.RequestHandlers.initHandlersFromConfig created /update/json: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 1011913 T1785 oasc.RequestHandlers.initHandlersFromConfig created /update/csv: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 1011914 T1785 oasc.RequestHandlers.initHandlersFromConfig created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 1011915 T1785 oasc.RequestHandlers.initHandlersFromConfig created /config: org.apache.solr.handler.SolrConfigHandler
   [junit4]   2> 1011915 T1785 oasc.RequestHandlers.initHandlersFromConfig created /schema: org.apache.solr.handler.SchemaHandler
   [junit4]   2> 1011917 T1785 oasc.RequestHandlers.initHandlersFromConfig created /replication: org.apache.solr.handler.ReplicationHandler
   [junit4]   2> 1011918 T1785 oasc.RequestHandlers.initHandlersFromConfig created /get: org.apache.solr.handler.RealTimeGetHandler
   [junit4]   2> 1011919 T1785 oasc.RequestHandlers.initHandlersFromConfig created /admin/luke: org.apache.solr.handler.admin.LukeRequestHandler
   [junit4]   2> 1011920 T1785 oasc.RequestHandlers.initHandlersFromConfig created /admin/system: org.apache.solr.handler.admin.SystemInfoHandler
   [junit4]   2> 1011921 T1785 oasc.RequestHandlers.initHandlersFromConfig created /admin/mbeans: org.apache.solr.handler.admin.SolrInfoMBeanHandler
   [junit4]   2> 1011921 T1785 oasc.RequestHandlers.initHandlersFromConfig created /admin/plugins: org.apache.solr.handler.admin.PluginInfoHandler
   [junit4]   2> 1011922 T1785 oasc.RequestHandlers.initHandlersFromConfig created /admin/threads: org.apache.solr.handler.admin.ThreadDumpHandler
   [junit4]   2> 1011923 T1785 oasc.RequestHandlers.initHandlersFromConfig created /admin/properties: org.apache.solr.handler.admin.PropertiesRequestHandler
   [junit4]   2> 1011924 T1785 oasc.RequestHandlers.initHandlersFromConfig created /admin/logging: org.apache.solr.handler.admin.LoggingHandler
   [junit4]   2> 1011924 T1785 oasc.RequestHandlers.initHandlersFromConfig created /admin/ping: org.apache.solr.handler.PingRequestHandler
   [junit4]   2> 1011925 T1785 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 1011926 T1785 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
   [junit4]   2> 1011926 T1785 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
   [junit4]   2> 1011948 T1785 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 1011950 T1785 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 1011953 T1785 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 1011955 T1785 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 1011967 T1785 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 1011967 T1785 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 1011968 T1785 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=11, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.6068313772277054]
   [junit4]   2> 1011970 T1785 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.cloud.ChaosMonkeySafeLeaderTest-A7D30E6BB32BE25E-001/tempDir-001/jetty1/index,segFN=segments_1,generation=1}
   [junit4]   2> 1011970 T1785 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 1011971 T1785 oass.SolrIndexSearcher.<init> Opening Searcher@7a4abc92[collection1] main
   [junit4]   2> 1011971 T1785 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
   [junit4]   2> 1011972 T1785 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1011972 T1785 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 1011973 T1785 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 1011973 T1785 oasr.RestManager.init Initializing RestManager with initArgs: {}
   [junit4]   2> 1011973 T1785 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1011974 T1785 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream Read 38 bytes from znode /configs/conf1/_rest_managed.json
   [junit4]   2> 1011974 T1785 oasr.ManagedResourceStorage.load Loaded LinkedHashMap at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1011975 T1785 oasr.ManagedResource.reloadFromStorage Loaded initArgs {} for /rest/managed
   [junit4]   2> 1011975 T1785 oasr.RestManager.init Initializing 0 registered ManagedResources
   [junit4]   2> 1011975 T1785 oash.ReplicationHandler.inform Commits will be reserved for  10000
   [junit4]   2> 1011976 T1786 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@7a4abc92[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1011977 T1785 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 1011978 T1789 oasc.ZkController.register Register replica - core:collection1 address:https://127.0.0.1:31330 collection:collection1 shard:shard3
   [junit4]   2> 1011978 T1728 oass.SolrDispatchFilter.init user.dir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2
   [junit4]   2> 1011979 T1728 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 1011979 T1789 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard3/election
   [junit4]   2> 1011983 T1789 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard3
   [junit4]   2> 1011985 T1757 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 1011985 T1789 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
   [junit4]   2> 1011985 T1789 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
   [junit4]   2> ASYNC  NEW_CORE C265 name=collection1 org.apache.solr.core.SolrCore@5859cd82 url=https://127.0.0.1:31330/collection1 node=127.0.0.1:31330_ C265_STATE=coll:collection1 core:collection1 props:{base_url=https://127.0.0.1:31330, node_name=127.0.0.1:31330_, core=collection1, state=down}
   [junit4]   2> 1011986 T1789 C265 P31330 oasc.SyncStrategy.sync Sync replicas to https://127.0.0.1:31330/collection1/
   [junit4]   2> 1011986 T1758 oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "operation":"leader",
   [junit4]   2> 	  "shard":"shard3",
   [junit4]   2> 	  "collection":"collection1"} current state version: 5
   [junit4]   2> 1011986 T1789 C265 P31330 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
   [junit4]   2> 1011986 T1789 C265 P31330 oasc.SyncStrategy.syncToMe https://127.0.0.1:31330/collection1/ has no replicas
   [junit4]   2> 1011986 T1789 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: https://127.0.0.1:31330/collection1/ shard3
   [junit4]   2> 1011987 T1789 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard3
   [junit4]   2> 1011987 T1757 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1011987 T1784 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1011987 T1768 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1011991 T1769 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 1011991 T1758 oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "operation":"leader",
   [junit4]   2> 	  "shard":"shard3",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "base_url":"https://127.0.0.1:31330",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "state":"active"} current state version: 6
   [junit4]   2> 1012094 T1768 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1012094 T1784 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1012143 T1789 oasc.ZkController.register We are https://127.0.0.1:31330/collection1/ and leader is https://127.0.0.1:31330/collection1/
   [junit4]   2> 1012144 T1789 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:31330
   [junit4]   2> 1012144 T1789 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
   [junit4]   2> 1012144 T1789 oasc.ZkController.publish publishing core=collection1 state=active collection=collection1
   [junit4]   2> 1012144 T1789 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 1012146 T1757 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 1012147 T1758 oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "base_url":"https://127.0.0.1:31330",
   [junit4]   2> 	  "numShards":"3",
   [junit4]   2> 	  "core_node_name":"core_node1",
   [junit4]   2> 	  "shard":"shard3",
   [junit4]   2> 	  "node_name":"127.0.0.1:31330_",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "state":"active",
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "roles":null} current state version: 7
   [junit4]   2> 1012148 T1758 oasco.ReplicaMutator.updateState Update state numShards=3 message={
   [junit4]   2> 	  "base_url":"https://127.0.0.1:31330",
   [junit4]   2> 	  "numShards":"3",
   [junit4]   2> 	  "core_node_name":"core_node1",
   [junit4]   2> 	  "shard":"shard3",
   [junit4]   2> 	  "node_name":"127.0.0.1:31330_",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "state":"active",
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "roles":null}
   [junit4]   2> 1012251 T1768 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1012251 T1784 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1012251 T1757 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1013634 T1728 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 2
   [junit4]   2> 1013635 T1728 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 1013642 T1728 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
   [junit4]   2> 1013644 T1728 oejs.AbstractConnector.doStart Started SslSocketConnector@127.0.0.1:17052
   [junit4]   2> 1013647 T1728 oass.SolrDispatchFilter.init SolrDispatchFilter.init()sun.misc.Launcher$AppClassLoader@57bd06bf
   [junit4]   2> 1013647 T1728 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1013647 T1728 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.cloud.ChaosMonkeySafeLeaderTest-A7D30E6BB32BE25E-001/tempDir-004
   [junit4]   2> 1013648 T1728 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.cloud.ChaosMonkeySafeLeaderTest-A7D30E6BB32BE25E-001/tempDir-004/'
   [junit4]   2> 1013681 T1728 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.cloud.ChaosMonkeySafeLeaderTest-A7D30E6BB32BE25E-001/tempDir-004/solr.xml
   [junit4]   2> 1013742 T1728 oasc.CoreContainer.<init> New CoreContainer 1325098525
   [junit4]   2> 1013742 T1728 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.cloud.ChaosMonkeySafeLeaderTest-A7D30E6BB32BE25E-001/tempDir-004/]
   [junit4]   2> 1013743 T1728 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 1013743 T1728 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 1013744 T1728 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 1013744 T1728 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 1013744 T1728 oashc.HttpShardHandlerFactory.getParameter Setting maxConnections to: 10000
   [junit4]   2> 1013744 T1728 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 1013745 T1728 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 1013745 T1728 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 1013745 T1728 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 1013745 T1728 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 1013748 T1728 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
   [junit4]   2> 1013751 T1728 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1013751 T1728 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1013751 T1728 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 1013752 T1728 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:38889/solr
   [junit4]   2> 1013752 T1728 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 1013752 T1728 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 1013753 T1728 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1013755 T1801 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@729cde4d name:ZooKeeperConnection Watcher:127.0.0.1:38889 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1013756 T1728 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1013756 T1728 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 1013758 T1728 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1013759 T1804 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1292d034 name:ZooKeeperConnection Watcher:127.0.0.1:38889/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1013759 T1728 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1013764 T1728 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 1014767 T1728 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:17052_
   [junit4]   2> 1014768 T1728 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:17052_
   [junit4]   2> 1014771 T1728 oasc.Overseer.close Overseer (id=null) closing
   [junit4]   2> 1014779 T1805 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
   [junit4]   2> 1014779 T1805 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 1014780 T1757 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 1014780 T1805 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 1014781 T1758 oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "base_url":"https://127.0.0.1:17052",
   [junit4]   2> 	  "numShards":"3",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:17052_",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "roles":null} current state version: 8
   [junit4]   2> 1014782 T1758 oasco.ReplicaMutator.updateState Update state numShards=3 message={
   [junit4]   2> 	  "base_url":"https://127.0.0.1:17052",
   [junit4]   2> 	  "numShards":"3",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:17052_",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "roles":null}
   [junit4]   2> 1014782 T1758 oasco.ReplicaMutator.updateState Collection already exists with numShards=3
   [junit4]   2> 1014782 T1758 oasco.ReplicaMutator.updateState Assigning new node to shard shard=shard2
   [junit4]   2> 1014783 T1757 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 1014783 T1784 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 1014783 T1768 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 1014783 T1804 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 1015781 T1805 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1015782 T1805 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
   [junit4]   2> 1015782 T1805 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 1015783 T1805 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
   [junit4]   2> 1015783 T1805 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1015783 T1805 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.cloud.ChaosMonkeySafeLeaderTest-A7D30E6BB32BE25E-001/tempDir-004/collection1/'
   [junit4]   2> 1015785 T1805 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.cloud.ChaosMonkeySafeLeaderTest-A7D30E6BB32BE25E-001/tempDir-004/collection1/lib/.svn/' to classloader
   [junit4]   2> 1015785 T1805 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.cloud.ChaosMonkeySafeLeaderTest-A7D30E6BB32BE25E-001/tempDir-004/collection1/lib/classes/' to classloader
   [junit4]   2> 1015785 T1805 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.cloud.ChaosMonkeySafeLeaderTest-A7D30E6BB32BE25E-001/tempDir-004/collection1/lib/README' to classloader
   [junit4]   2> 1015808 T1805 oasc.ZkController.watchZKConfDir watch zkdir /configs/conf1
   [junit4]   2> 1015810 T1805 oasc.Config.<init> loaded config solrconfig.xml with version 0 
   [junit4]   2> 1015835 T1805 oasc.SolrConfig.<init> Using Lucene MatchVersion: 5.0.0
   [junit4]   2> 1015868 T1805 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1015970 T1805 oass.IndexSchema.readSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 1015977 T1805 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 1016308 T1805 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 1016311 T1805 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 1016313 T1805 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 1016318 T1805 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 1016338 T1805 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection collection1
   [junit4]   2> 1016338 T1805 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
   [junit4]   2> 1016339 T1805 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.cloud.ChaosMonkeySafeLeaderTest-A7D30E6BB32BE25E-001/tempDir-004/collection1/, dataDir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.cloud.ChaosMonkeySafeLeaderTest-A7D30E6BB32BE25E-001/tempDir-001/jetty2/
   [junit4]   2> 1016339 T1805 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@5089dee5
   [junit4]   2> 1016341 T1805 oasc.CachingDirectoryFactory.get return new directory for /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.cloud.ChaosMonkeySafeLeaderTest-A7D30E6BB32BE25E-001/tempDir-001/jetty2
   [junit4]   2> 1016341 T1805 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.cloud.ChaosMonkeySafeLeaderTest-A7D30E6BB32BE25E-001/tempDir-001/jetty2/index/
   [junit4]   2> 1016342 T1805 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.cloud.ChaosMonkeySafeLeaderTest-A7D30E6BB32BE25E-001/tempDir-001/jetty2/index' doesn't exist. Creating new index...
   [junit4]   2> 1016344 T1805 oasc.CachingDirectoryFactory.get return new directory for /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.cloud.ChaosMonkeySafeLeaderTest-A7D30E6BB32BE25E-001/tempDir-001/jetty2/index
   [junit4]   2> 1016344 T1805 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=48, maxMergeAtOnceExplicit=23, maxMergedSegmentMB=85.8037109375, floorSegmentMB=0.783203125, forceMergeDeletesPctAllowed=24.188309162155292, segmentsPerTier=33.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 1016350 T1805 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.cloud.ChaosMonkeySafeLeaderTest-A7D30E6BB32BE25E-001/tempDir-001/jetty2/index,segFN=segments_1,generation=1}
   [junit4]   2> 1016350 T1805 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 1016358 T1805 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 1016358 T1805 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 1016358 T1805 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 1016359 T1805 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1016359 T1805 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1016359 T1805 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 1016360 T1805 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1016360 T1805 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1016360 T1805 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 1016361 T1805 oasc.RequestHandlers.initHandlersFromConfig created /update: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 1016362 T1805 oasc.RequestHandlers.initHandlersFromConfig created /update/json: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 1016362 T1805 oasc.RequestHandlers.initHandlersFromConfig created /update/csv: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 1016363 T1805 oasc.RequestHandlers.initHandlersFromConfig created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 1016364 T1805 oasc.RequestHandlers.initHandlersFromConfig created /config: org.apache.solr.handler.SolrConfigHandler
   [junit4]   2> 1016364 T1805 oasc.RequestHandlers.initHandlersFromConfig created /schema: org.apache.solr.handler.SchemaHandler
   [junit4]   2> 1016366 T1805 oasc.RequestHandlers.initHandlersFromConfig created /replication: org.apache.solr.handler.ReplicationHandler
   [junit4]   2> 1016367 T1805 oasc.RequestHandlers.initHandlersFromConfig created /get: org.apache.solr.handler.RealTimeGetHandler
   [junit4]   2> 1016368 T1805 oasc.RequestHandlers.initHandlersFromConfig created /admin/luke: org.apache.solr.handler.admin.LukeRequestHandler
   [junit4]   2> 1016369 T1805 oasc.RequestHandlers.initHandlersFromConfig created /admin/system: org.apache.solr.handler.admin.SystemInfoHandler
   [junit4]   2> 1016372 T1805 oasc.RequestHandlers.initHandlersFromConfig created /admin/mbeans: org.apache.solr.handler.admin.SolrInfoMBeanHandler
   [junit4]   2> 1016374 T1805 oasc.RequestHandlers.initHandlersFromConfig created /admin/plugins: org.apache.solr.handler.admin.PluginInfoHandler
   [junit4]   2> 1016375 T1805 oasc.RequestHandlers.initHandlersFromConfig created /admin/threads: org.apache.solr.handler.admin.ThreadDumpHandler
   [junit4]   2> 1016377 T1805 oasc.RequestHandlers.initHandlersFromConfig created /admin/properties: org.apache.solr.handler.admin.PropertiesRequestHandler
   [junit4]   2> 1016378 T1805 oasc.RequestHandlers.initHandlersFromConfig created /admin/logging: org.apache.solr.handler.admin.LoggingHandler
   [junit4]   2> 1016379 T1805 oasc.RequestHandlers.initHandlersFromConfig created /admin/ping: org.apache.solr.handler.PingRequestHandler
   [junit4]   2> 1016380 T1805 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 1016381 T1805 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
   [junit4]   2> 1016382 T1805 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
   [junit4]   2> 1016401 T1805 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 1016403 T1805 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 1016405 T1805 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 1016407 T1805 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 1016411 T1805 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 1016411 T1805 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 1016413 T1805 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=11, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.6068313772277054]
   [junit4]   2> 1016414 T1805 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.cloud.ChaosMonkeySafeLeaderTest-A7D30E6BB32BE25E-001/tempDir-001/jetty2/index,segFN=segments_1,generation=1}
   [junit4]   2> 1016415 T1805 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 1016415 T1805 oass.SolrIndexSearcher.<init> Opening Searcher@4d5ddf89[collection1] main
   [junit4]   2> 1016415 T1805 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
   [junit4]   2> 1016416 T1805 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1016417 T1805 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 1016417 T1805 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 1016417 T1805 oasr.RestManager.init Initializing RestManager with initArgs: {}
   [junit4]   2> 1016418 T1805 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1016418 T1805 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream Read 38 bytes from znode /configs/conf1/_rest_managed.json
   [junit4]   2> 1016419 T1805 oasr.ManagedResourceStorage.load Loaded LinkedHashMap at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1016419 T1805 oasr.ManagedResource.reloadFromStorage Loaded initArgs {} for /rest/managed
   [junit4]   2> 1016419 T1805 oasr.RestManager.init Initializing 0 registered ManagedResources
   [junit4]   2> 1016419 T1805 oash.ReplicationHandler.inform Commits will be reserved for  10000
   [junit4]   2> 1016421 T1806 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4d5ddf89[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1016421 T1805 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 1016422 T1809 oasc.ZkController.register Register replica - core:collection1 address:https://127.0.0.1:17052 collection:collection1 shard:shard2
   [junit4]   2> 1016422 T1728 oass.SolrDispatchFilter.init user.dir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2
   [junit4]   2> 1016423 T1728 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 1016423 T1809 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
   [junit4]   2> 1016427 T1809 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard2
   [junit4]   2> 1016429 T1757 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 1016429 T1809 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
   [junit4]   2> 1016429 T1809 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
   [junit4]   2> ASYNC  NEW_CORE C266 name=collection1 org.apache.solr.core.SolrCore@687253f1 url=https://127.0.0.1:17052/collection1 node=127.0.0.1:17052_ C266_STATE=coll:collection1 core:collection1 props:{base_url=https://127.0.0.1:17052, node_name=127.0.0.1:17052_, core=collection1, state=down}
   [junit4]   2> 1016429 T1809 C266 P17052 oasc.SyncStrategy.sync Sync replicas to https://127.0.0.1:17052/collection1/
   [junit4]   2> 1016429 T1758 oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "operation":"leader",
   [junit4]   2> 	  "shard":"shard2",
   [junit4]   2> 	  "collection":"collection1"} current state version: 9
   [junit4]   2> 1016430 T1809 C266 P17052 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
   [junit4]   2> 1016430 T1809 C266 P17052 oasc.SyncStrategy.syncToMe https://127.0.0.1:17052/collection1/ has no replicas
   [junit4]   2> 1016430 T1809 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: https://127.0.0.1:17052/collection1/ shard2
   [junit4]   2> 1016431 T1809 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
   [junit4]   2> 1016431 T1784 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 1016431 T1768 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 1016431 T1757 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 1016431 T1804 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 1016434 T1758 oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "operation":"leader",
   [junit4]   2> 	  "shard":"shard2",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "base_url":"https://127.0.0.1:17052",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "state":"active"} current state version: 10
   [junit4]   2> 1016436 T1769 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 1016437 T1768 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 1016437 T1804 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 1016437 T1784 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 1016486 T1809 oasc.ZkController.register We are https://127.0.0.1:17052/collection1/ and leader is https://127.0.0.1:17052/collection1/
   [junit4]   2> 1016487 T1809 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:17052
   [junit4]   2> 1016487 T1809 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
   [junit4]   2> 1016487 T1809 oasc.ZkController.publish publishing core=collection1 state=active collection=collection1
   [junit4]   2> 1016487 T1809 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 1016489 T1757 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 1016490 T1758 oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "base_url":"https://127.0.0.1:17052",
   [junit4]   2> 	  "numShards":"3",
   [junit4]   2> 	  "core_node_name":"core_node2",
   [junit4]   2> 	  "shard":"shard2",
   [junit4]   2> 	  "node_name":"127.0.0.1:17052_",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "state":"active",
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "roles":null} current state version: 11
   [junit4]   2> 1016490 T1758 oasco.ReplicaMutator.updateState Update state numShards=3 messag

[...truncated too long message...]

ack.transfer(SynchronousQueue.java:359)
   [junit4]   2> 	        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)
   [junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]   2> 	        at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 	  18) Thread[id=1809, name=coreZkRegister-1025-thread-1, state=WAITING, group=TGRP-ChaosMonkeySafeLeaderTest]
   [junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
   [junit4]   2> 	        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
   [junit4]   2> 	        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]   2> 	        at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 	  19) Thread[id=1822, name=TEST-ChaosMonkeySafeLeaderTest.testDistribSearch-seed#[A7D30E6BB32BE25E]-SendThread(127.0.0.1:38889), state=RUNNABLE, group=TGRP-ChaosMonkeySafeLeaderTest]
   [junit4]   2> 	        at sun.nio.ch.IOUtil.drain(Native Method)
   [junit4]   2> 	        at sun.nio.ch.KQueueSelectorImpl.updateSelectedKeys(KQueueSelectorImpl.java:167)
   [junit4]   2> 	        at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:108)
   [junit4]   2> 	        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
   [junit4]   2> 	        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
   [junit4]   2> 	        at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:349)
   [junit4]   2> 	        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
   [junit4]   2> 	  20) Thread[id=1827, name=Thread-1083, state=WAITING, group=TGRP-ChaosMonkeySafeLeaderTest]
   [junit4]   2> 	        at java.lang.Object.wait(Native Method)
   [junit4]   2> 	        at java.lang.Object.wait(Object.java:503)
   [junit4]   2> 	        at org.apache.solr.core.CloserThread.run(CoreContainer.java:929)
   [junit4]   2> 	  21) Thread[id=1849, name=coreZkRegister-1043-thread-1, state=WAITING, group=TGRP-ChaosMonkeySafeLeaderTest]
   [junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
   [junit4]   2> 	        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
   [junit4]   2> 	        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]   2> 	        at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 	  22) Thread[id=1790, name=HashSessionScavenger-99, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderTest]
   [junit4]   2> 	        at java.lang.Object.wait(Native Method)
   [junit4]   2> 	        at java.util.TimerThread.mainLoop(Timer.java:552)
   [junit4]   2> 	        at java.util.TimerThread.run(Timer.java:505)
   [junit4]   2> 	  23) Thread[id=1842, name=TEST-ChaosMonkeySafeLeaderTest.testDistribSearch-seed#[A7D30E6BB32BE25E]-SendThread(127.0.0.1:38889), state=RUNNABLE, group=TGRP-ChaosMonkeySafeLeaderTest]
   [junit4]   2> 	        at sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method)
   [junit4]   2> 	        at sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200)
   [junit4]   2> 	        at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103)
   [junit4]   2> 	        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
   [junit4]   2> 	        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
   [junit4]   2> 	        at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:349)
   [junit4]   2> 	        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
   [junit4]   2> 	  24) Thread[id=1857, name=commitScheduler-1033-thread-1, state=WAITING, group=TGRP-ChaosMonkeySafeLeaderTest]
   [junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
   [junit4]   2> 	        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
   [junit4]   2> 	        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1079)
   [junit4]   2> 	        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)
   [junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]   2> 	        at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 	  25) Thread[id=1846, name=searcherExecutor-1048-thread-1, state=WAITING, group=TGRP-ChaosMonkeySafeLeaderTest]
   [junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
   [junit4]   2> 	        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
   [junit4]   2> 	        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]   2> 	        at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> NOTE: test params are: codec=Lucene50, sim=RandomSimilarityProvider(queryNorm=true,coord=crazy): {}, locale=en_PH, timezone=Europe/Warsaw
   [junit4]   2> NOTE: FreeBSD 9.1-RELEASE-p3 amd64/Oracle Corporation 1.7.0_65 (64-bit)/cpus=16,threads=28,free=118919648,total=168296448
   [junit4]   2> NOTE: All tests run in this JVM: [AsyncMigrateRouteKeyTest, RollingRestartTest, TestLFUCache, RankQueryTest, TestSolrJ, TestSimpleQParserPlugin, TestFieldTypeResource, TestReplicationHandler, RemoteQueryErrorTest, PreAnalyzedUpdateProcessorTest, DistributedSuggestComponentTest, TestIntervalFaceting, TestRestManager, TestPivotHelperCode, TestManagedSynonymFilterFactory, StandardRequestHandlerTest, CachingDirectoryFactoryTest, TestNonNRTOpen, TestDistributedMissingSort, TestManagedSchemaFieldResource, TestFastWriter, ReturnFieldsTest, ResourceLoaderTest, TestSuggestSpellingConverter, CircularListTest, TestInitQParser, ChaosMonkeySafeLeaderTest]
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=ChaosMonkeySafeLeaderTest -Dtests.seed=A7D30E6BB32BE25E -Dtests.multiplier=2 -Dtests.slow=true -Dtests.locale=en_PH -Dtests.timezone=Europe/Warsaw -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
   [junit4] ERROR   0.00s J2 | ChaosMonkeySafeLeaderTest (suite) <<<
   [junit4]    > Throwable #1: java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([A7D30E6BB32BE25E]:0)
   [junit4] Completed on J2 in 7223.83s, 1 test, 2 errors <<< FAILURES!

[...truncated 103 lines...]
BUILD FAILED
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/build.xml:529: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/build.xml:477: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/build.xml:61: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/extra-targets.xml:39: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build.xml:190: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/common-build.xml:508: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/lucene/common-build.xml:1363: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/lucene/common-build.xml:970: There were test failures: 453 suites, 1853 tests, 1 suite-level error, 1 error, 145 ignored (23 assumptions)

Total time: 157 minutes 24 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Sending artifact delta relative to Lucene-Solr-Tests-5.x-Java7 #2368
Archived 5 artifacts
Archive block size is 32768
Received 0 blocks and 187490492 bytes
Compression is 0.0%
Took 37 sec
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure