You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Policeman Jenkins Server <je...@thetaphi.de> on 2013/03/03 22:44:22 UTC
[JENKINS] Lucene-Solr-trunk-Linux (32bit/jdk1.7.0_15) - Build #
4557 - Failure!
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/4557/
Java: 32bit/jdk1.7.0_15 -server -XX:+UseSerialGC
All tests passed
Build Log:
[...truncated 24207 lines...]
-check-forbidden-java-apis:
[forbidden-apis] Reading bundled API signatures: jdk-unsafe-1.6
[forbidden-apis] Reading bundled API signatures: jdk-deprecated-1.6
[forbidden-apis] Reading bundled API signatures: commons-io-unsafe-2.1
[forbidden-apis] Reading API signatures: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/tools/forbiddenApis/executors.txt
[forbidden-apis] Reading API signatures: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/tools/forbiddenApis/servlet-api.txt
[forbidden-apis] Loading classes to check...
[forbidden-apis] Scanning for API signatures and dependencies...
[forbidden-apis] WARNING: The referenced class 'org.apache.lucene.analysis.uima.ae.AEProviderFactory' cannot be loaded. Please fix the classpath!
[forbidden-apis] WARNING: The referenced class 'org.apache.lucene.analysis.uima.ae.AEProviderFactory' cannot be loaded. Please fix the classpath!
[forbidden-apis] WARNING: The referenced class 'org.apache.lucene.analysis.uima.ae.AEProvider' cannot be loaded. Please fix the classpath!
[forbidden-apis] WARNING: The referenced class 'org.apache.lucene.collation.ICUCollationKeyAnalyzer' cannot be loaded. Please fix the classpath!
[forbidden-apis] Forbidden class/interface use: java.io.FileWriter [Uses default charset]
[forbidden-apis] in org.apache.solr.core.TestSolrDiscoveryProperties (TestSolrDiscoveryProperties.java:60)
[forbidden-apis] Forbidden class/interface use: java.io.FileWriter [Uses default charset]
[forbidden-apis] in org.apache.solr.core.TestSolrDiscoveryProperties (TestSolrDiscoveryProperties.java:62)
[forbidden-apis] Forbidden class/interface use: java.io.FileWriter [Uses default charset]
[forbidden-apis] in org.apache.solr.core.TestSolrDiscoveryProperties (TestSolrDiscoveryProperties.java:94)
[forbidden-apis] Forbidden class/interface use: java.io.FileWriter [Uses default charset]
[forbidden-apis] in org.apache.solr.core.TestSolrDiscoveryProperties (TestSolrDiscoveryProperties.java:96)
[forbidden-apis] Scanned 2217 (and 1249 related) class file(s) for forbidden API invocations (in 1.79s), 4 error(s).
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:381: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:67: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build.xml:265: Check for forbidden API calls failed, see log.
Total time: 47 minutes 37 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Description set: Java: 32bit/jdk1.7.0_15 -server -XX:+UseSerialGC
Email was triggered for: Failure
Sending email for trigger: Failure
Re: [JENKINS] Lucene-Solr-trunk-Linux (32bit/jdk1.7.0_15) - Build #
4557 - Failure!
Posted by Erick Erickson <er...@gmail.com>.
Never mind, Uwe beat me to it.
On Sun, Mar 3, 2013 at 6:54 PM, Erick Erickson <er...@gmail.com>wrote:
> Gaaah, that'll teach me to try to go really fast. I'll fix momentarily.
>
>
>
> On Sun, Mar 3, 2013 at 4:44 PM, Policeman Jenkins Server <
> jenkins@thetaphi.de> wrote:
>
>> Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/4557/
>> Java: 32bit/jdk1.7.0_15 -server -XX:+UseSerialGC
>>
>> All tests passed
>>
>> Build Log:
>> [...truncated 24207 lines...]
>> -check-forbidden-java-apis:
>> [forbidden-apis] Reading bundled API signatures: jdk-unsafe-1.6
>> [forbidden-apis] Reading bundled API signatures: jdk-deprecated-1.6
>> [forbidden-apis] Reading bundled API signatures: commons-io-unsafe-2.1
>> [forbidden-apis] Reading API signatures:
>> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/tools/forbiddenApis/executors.txt
>> [forbidden-apis] Reading API signatures:
>> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/tools/forbiddenApis/servlet-api.txt
>> [forbidden-apis] Loading classes to check...
>> [forbidden-apis] Scanning for API signatures and dependencies...
>> [forbidden-apis] WARNING: The referenced class
>> 'org.apache.lucene.analysis.uima.ae.AEProviderFactory' cannot be loaded.
>> Please fix the classpath!
>> [forbidden-apis] WARNING: The referenced class
>> 'org.apache.lucene.analysis.uima.ae.AEProviderFactory' cannot be loaded.
>> Please fix the classpath!
>> [forbidden-apis] WARNING: The referenced class
>> 'org.apache.lucene.analysis.uima.ae.AEProvider' cannot be loaded. Please
>> fix the classpath!
>> [forbidden-apis] WARNING: The referenced class
>> 'org.apache.lucene.collation.ICUCollationKeyAnalyzer' cannot be loaded.
>> Please fix the classpath!
>> [forbidden-apis] Forbidden class/interface use: java.io.FileWriter [Uses
>> default charset]
>> [forbidden-apis] in org.apache.solr.core.TestSolrDiscoveryProperties
>> (TestSolrDiscoveryProperties.java:60)
>> [forbidden-apis] Forbidden class/interface use: java.io.FileWriter [Uses
>> default charset]
>> [forbidden-apis] in org.apache.solr.core.TestSolrDiscoveryProperties
>> (TestSolrDiscoveryProperties.java:62)
>> [forbidden-apis] Forbidden class/interface use: java.io.FileWriter [Uses
>> default charset]
>> [forbidden-apis] in org.apache.solr.core.TestSolrDiscoveryProperties
>> (TestSolrDiscoveryProperties.java:94)
>> [forbidden-apis] Forbidden class/interface use: java.io.FileWriter [Uses
>> default charset]
>> [forbidden-apis] in org.apache.solr.core.TestSolrDiscoveryProperties
>> (TestSolrDiscoveryProperties.java:96)
>> [forbidden-apis] Scanned 2217 (and 1249 related) class file(s) for
>> forbidden API invocations (in 1.79s), 4 error(s).
>>
>> BUILD FAILED
>> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:381: The
>> following error occurred while executing this line:
>> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:67: The
>> following error occurred while executing this line:
>> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build.xml:265:
>> Check for forbidden API calls failed, see log.
>>
>> Total time: 47 minutes 37 seconds
>> Build step 'Invoke Ant' marked build as failure
>> Archiving artifacts
>> Recording test results
>> Description set: Java: 32bit/jdk1.7.0_15 -server -XX:+UseSerialGC
>> Email was triggered for: Failure
>> Sending email for trigger: Failure
>>
>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: dev-help@lucene.apache.org
>>
>
>
Re: [JENKINS] Lucene-Solr-trunk-Linux (32bit/jdk1.7.0_15) - Build #
4557 - Failure!
Posted by Erick Erickson <er...@gmail.com>.
Gaaah, that'll teach me to try to go really fast. I'll fix momentarily.
On Sun, Mar 3, 2013 at 4:44 PM, Policeman Jenkins Server <
jenkins@thetaphi.de> wrote:
> Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/4557/
> Java: 32bit/jdk1.7.0_15 -server -XX:+UseSerialGC
>
> All tests passed
>
> Build Log:
> [...truncated 24207 lines...]
> -check-forbidden-java-apis:
> [forbidden-apis] Reading bundled API signatures: jdk-unsafe-1.6
> [forbidden-apis] Reading bundled API signatures: jdk-deprecated-1.6
> [forbidden-apis] Reading bundled API signatures: commons-io-unsafe-2.1
> [forbidden-apis] Reading API signatures:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/tools/forbiddenApis/executors.txt
> [forbidden-apis] Reading API signatures:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/tools/forbiddenApis/servlet-api.txt
> [forbidden-apis] Loading classes to check...
> [forbidden-apis] Scanning for API signatures and dependencies...
> [forbidden-apis] WARNING: The referenced class
> 'org.apache.lucene.analysis.uima.ae.AEProviderFactory' cannot be loaded.
> Please fix the classpath!
> [forbidden-apis] WARNING: The referenced class
> 'org.apache.lucene.analysis.uima.ae.AEProviderFactory' cannot be loaded.
> Please fix the classpath!
> [forbidden-apis] WARNING: The referenced class
> 'org.apache.lucene.analysis.uima.ae.AEProvider' cannot be loaded. Please
> fix the classpath!
> [forbidden-apis] WARNING: The referenced class
> 'org.apache.lucene.collation.ICUCollationKeyAnalyzer' cannot be loaded.
> Please fix the classpath!
> [forbidden-apis] Forbidden class/interface use: java.io.FileWriter [Uses
> default charset]
> [forbidden-apis] in org.apache.solr.core.TestSolrDiscoveryProperties
> (TestSolrDiscoveryProperties.java:60)
> [forbidden-apis] Forbidden class/interface use: java.io.FileWriter [Uses
> default charset]
> [forbidden-apis] in org.apache.solr.core.TestSolrDiscoveryProperties
> (TestSolrDiscoveryProperties.java:62)
> [forbidden-apis] Forbidden class/interface use: java.io.FileWriter [Uses
> default charset]
> [forbidden-apis] in org.apache.solr.core.TestSolrDiscoveryProperties
> (TestSolrDiscoveryProperties.java:94)
> [forbidden-apis] Forbidden class/interface use: java.io.FileWriter [Uses
> default charset]
> [forbidden-apis] in org.apache.solr.core.TestSolrDiscoveryProperties
> (TestSolrDiscoveryProperties.java:96)
> [forbidden-apis] Scanned 2217 (and 1249 related) class file(s) for
> forbidden API invocations (in 1.79s), 4 error(s).
>
> BUILD FAILED
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:381: The
> following error occurred while executing this line:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:67: The
> following error occurred while executing this line:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build.xml:265:
> Check for forbidden API calls failed, see log.
>
> Total time: 47 minutes 37 seconds
> Build step 'Invoke Ant' marked build as failure
> Archiving artifacts
> Recording test results
> Description set: Java: 32bit/jdk1.7.0_15 -server -XX:+UseSerialGC
> Email was triggered for: Failure
> Sending email for trigger: Failure
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
Re: [JENKINS] Lucene-Solr-trunk-Linux (32bit/jdk1.8.0-ea-b79) - Build # 4558 - Still Failing!
Posted by Mark Miller <ma...@gmail.com>.
No time at the moment, but this one should prob be tracked down before 4.2
- Mark
On Mar 3, 2013, at 3:43 PM, Policeman Jenkins Server <je...@thetaphi.de> wrote:
> Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/4558/
> Java: 32bit/jdk1.8.0-ea-b79 -client -XX:+UseSerialGC
>
> 2 tests failed.
> REGRESSION: org.apache.solr.cloud.BasicDistributedZk2Test.testDistribSearch
>
> Error Message:
> Server at http://127.0.0.1:49875/ds_/onenodecollectioncore returned non ok status:404, message:Can not find: /ds_/onenodecollectioncore/update
>
> Stack Trace:
> org.apache.solr.common.SolrException: Server at http://127.0.0.1:49875/ds_/onenodecollectioncore returned non ok status:404, message:Can not find: /ds_/onenodecollectioncore/update
> at __randomizedtesting.SeedInfo.seed([E33CD887C055A93A:62DA569FB70AC906]:0)
> at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:375)
> at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:181)
> at org.apache.solr.client.solrj.request.AbstractUpdateRequest.process(AbstractUpdateRequest.java:117)
> at org.apache.solr.client.solrj.SolrServer.add(SolrServer.java:116)
> at org.apache.solr.client.solrj.SolrServer.add(SolrServer.java:102)
> at org.apache.solr.cloud.BasicDistributedZk2Test.testNodeWithoutCollectionForwarding(BasicDistributedZk2Test.java:200)
> at org.apache.solr.cloud.BasicDistributedZk2Test.doTest(BasicDistributedZk2Test.java:89)
> at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:806)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:487)
> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
> at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
> at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
> at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
> at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
> at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
> at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
> at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
> at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
> at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
> at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
> at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
> at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
> at java.lang.Thread.run(Thread.java:722)
>
>
> REGRESSION: org.apache.solr.cloud.UnloadDistributedZkTest.testDistribSearch
>
> Error Message:
> Still found shard
>
> Stack Trace:
> java.lang.AssertionError: Still found shard
> at __randomizedtesting.SeedInfo.seed([E33CD887C055A93A:62DA569FB70AC906]:0)
> at org.junit.Assert.fail(Assert.java:93)
> at org.apache.solr.cloud.UnloadDistributedZkTest.testUnloadShardAndCollection(UnloadDistributedZkTest.java:124)
> at org.apache.solr.cloud.UnloadDistributedZkTest.doTest(UnloadDistributedZkTest.java:77)
> at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:806)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:487)
> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
> at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
> at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
> at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
> at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
> at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
> at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
> at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
> at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
> at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
> at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
> at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
> at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
> at java.lang.Thread.run(Thread.java:722)
>
>
>
>
> Build Log:
> [...truncated 9183 lines...]
> [junit4:junit4] Suite: org.apache.solr.cloud.BasicDistributedZk2Test
> [junit4:junit4] 2> 0 T1132 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /ds_/
> [junit4:junit4] 2> 3 T1132 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
> [junit4:junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1362353595885
> [junit4:junit4] 2> 3 T1132 oasc.ZkTestServer.run STARTING ZK TEST SERVER
> [junit4:junit4] 2> 4 T1133 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
> [junit4:junit4] 2> 4 T1133 oazs.ZooKeeperServer.setTickTime tickTime set to 1000
> [junit4:junit4] 2> 5 T1133 oazs.ZooKeeperServer.setMinSessionTimeout minSessionTimeout set to -1
> [junit4:junit4] 2> 5 T1133 oazs.ZooKeeperServer.setMaxSessionTimeout maxSessionTimeout set to -1
> [junit4:junit4] 2> 5 T1133 oazs.NIOServerCnxnFactory.configure binding to port 0.0.0.0/0.0.0.0:0
> [junit4:junit4] 2> 6 T1133 oazsp.FileTxnSnapLog.save Snapshotting: 0x0 to /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/zookeeper/server1/data/version-2/snapshot.0
> [junit4:junit4] 2> 104 T1132 oasc.ZkTestServer.run start zk server on port:53275
> [junit4:junit4] 2> 104 T1132 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:53275 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@1b9eccd
> [junit4:junit4] 2> 105 T1132 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4:junit4] 2> 105 T1138 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:53275. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
> [junit4:junit4] 2> 106 T1138 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:53275, initiating session
> [junit4:junit4] 2> 106 T1134 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:43968
> [junit4:junit4] 2> 106 T1134 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:43968
> [junit4:junit4] 2> 106 T1136 oazsp.FileTxnLog.append Creating new log file: log.1
> [junit4:junit4] 2> 126 T1136 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d329a25f00000 with negotiated timeout 10000 for client /127.0.0.1:43968
> [junit4:junit4] 2> 126 T1138 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:53275, sessionid = 0x13d329a25f00000, negotiated timeout = 10000
> [junit4:junit4] 2> 127 T1139 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1b9eccd name:ZooKeeperConnection Watcher:127.0.0.1:53275 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4:junit4] 2> 127 T1132 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4:junit4] 2> 127 T1132 oascc.SolrZkClient.makePath makePath: /solr
> [junit4:junit4] 2> 130 T1137 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d329a25f00000
> [junit4:junit4] 2> 130 T1139 oaz.ClientCnxn$EventThread.run EventThread shut down
> [junit4:junit4] 2> 131 T1134 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:43968 which had sessionid 0x13d329a25f00000
> [junit4:junit4] 2> 130 T1132 oaz.ZooKeeper.close Session: 0x13d329a25f00000 closed
> [junit4:junit4] 2> 131 T1132 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:53275/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@17e1b0a
> [junit4:junit4] 2> 132 T1132 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4:junit4] 2> 132 T1140 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:53275. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
> [junit4:junit4] 2> 133 T1140 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:53275, initiating session
> [junit4:junit4] 2> 133 T1134 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:43969
> [junit4:junit4] 2> 133 T1134 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:43969
> [junit4:junit4] 2> 133 T1136 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d329a25f00001 with negotiated timeout 10000 for client /127.0.0.1:43969
> [junit4:junit4] 2> 133 T1140 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:53275, sessionid = 0x13d329a25f00001, negotiated timeout = 10000
> [junit4:junit4] 2> 134 T1141 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@17e1b0a name:ZooKeeperConnection Watcher:127.0.0.1:53275/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4:junit4] 2> 134 T1132 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4:junit4] 2> 134 T1132 oascc.SolrZkClient.makePath makePath: /collections/collection1
> [junit4:junit4] 2> 136 T1132 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
> [junit4:junit4] 2> 137 T1132 oascc.SolrZkClient.makePath makePath: /collections/control_collection
> [junit4:junit4] 2> 138 T1132 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
> [junit4:junit4] 2> 140 T1132 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
> [junit4:junit4] 2> 140 T1132 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
> [junit4:junit4] 2> 143 T1132 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
> [junit4:junit4] 2> 143 T1132 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
> [junit4:junit4] 2> 146 T1132 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
> [junit4:junit4] 2> 146 T1132 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
> [junit4:junit4] 2> 147 T1132 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
> [junit4:junit4] 2> 148 T1132 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
> [junit4:junit4] 2> 149 T1132 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
> [junit4:junit4] 2> 150 T1132 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
> [junit4:junit4] 2> 152 T1132 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
> [junit4:junit4] 2> 152 T1132 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
> [junit4:junit4] 2> 153 T1132 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
> [junit4:junit4] 2> 154 T1132 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
> [junit4:junit4] 2> 158 T1132 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
> [junit4:junit4] 2> 159 T1132 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
> [junit4:junit4] 2> 161 T1132 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
> [junit4:junit4] 2> 161 T1132 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
> [junit4:junit4] 2> 163 T1137 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d329a25f00001
> [junit4:junit4] 2> 163 T1141 oaz.ClientCnxn$EventThread.run EventThread shut down
> [junit4:junit4] 2> 163 T1134 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:43969 which had sessionid 0x13d329a25f00001
> [junit4:junit4] 2> 163 T1132 oaz.ZooKeeper.close Session: 0x13d329a25f00001 closed
> [junit4:junit4] 2> 222 T1132 oejs.Server.doStart jetty-8.1.8.v20121106
> [junit4:junit4] 2> 249 T1132 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:35276
> [junit4:junit4] 2> 250 T1132 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
> [junit4:junit4] 2> 250 T1132 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
> [junit4:junit4] 2> 251 T1132 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1362353596046
> [junit4:junit4] 2> 251 T1132 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1362353596046/solr.xml
> [junit4:junit4] 2> 251 T1132 oasc.CoreContainer.<init> New CoreContainer 20508092
> [junit4:junit4] 2> 252 T1132 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1362353596046/'
> [junit4:junit4] 2> 252 T1132 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1362353596046/'
> [junit4:junit4] 2> 269 T1132 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
> [junit4:junit4] 2> 270 T1132 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
> [junit4:junit4] 2> 270 T1132 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
> [junit4:junit4] 2> 270 T1132 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
> [junit4:junit4] 2> 271 T1132 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
> [junit4:junit4] 2> 271 T1132 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
> [junit4:junit4] 2> 271 T1132 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
> [junit4:junit4] 2> 272 T1132 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
> [junit4:junit4] 2> 272 T1132 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
> [junit4:junit4] 2> 272 T1132 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
> [junit4:junit4] 2> 279 T1132 oasc.CoreContainer.load Registering Log Listener
> [junit4:junit4] 2> 296 T1132 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:53275/solr
> [junit4:junit4] 2> 297 T1132 oasc.ZkController.checkChrootPath zkHost includes chroot
> [junit4:junit4] 2> 297 T1132 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:53275 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@10883c6
> [junit4:junit4] 2> 298 T1151 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:53275. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
> [junit4:junit4] 2> 298 T1132 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4:junit4] 2> 299 T1151 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:53275, initiating session
> [junit4:junit4] 2> 299 T1134 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:43970
> [junit4:junit4] 2> 299 T1134 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:43970
> [junit4:junit4] 2> 300 T1136 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d329a25f00002 with negotiated timeout 20000 for client /127.0.0.1:43970
> [junit4:junit4] 2> 300 T1151 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:53275, sessionid = 0x13d329a25f00002, negotiated timeout = 20000
> [junit4:junit4] 2> 300 T1152 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@10883c6 name:ZooKeeperConnection Watcher:127.0.0.1:53275 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4:junit4] 2> 300 T1132 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4:junit4] 2> 301 T1137 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d329a25f00002
> [junit4:junit4] 2> 302 T1152 oaz.ClientCnxn$EventThread.run EventThread shut down
> [junit4:junit4] 2> 302 T1134 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:43970 which had sessionid 0x13d329a25f00002
> [junit4:junit4] 2> 302 T1132 oaz.ZooKeeper.close Session: 0x13d329a25f00002 closed
> [junit4:junit4] 2> 303 T1132 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
> [junit4:junit4] 2> 305 T1132 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:53275/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@12a91bf
> [junit4:junit4] 2> 306 T1153 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:53275. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
> [junit4:junit4] 2> 306 T1132 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4:junit4] 2> 306 T1153 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:53275, initiating session
> [junit4:junit4] 2> 306 T1134 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:43971
> [junit4:junit4] 2> 307 T1134 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:43971
> [junit4:junit4] 2> 307 T1136 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d329a25f00003 with negotiated timeout 20000 for client /127.0.0.1:43971
> [junit4:junit4] 2> 307 T1153 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:53275, sessionid = 0x13d329a25f00003, negotiated timeout = 20000
> [junit4:junit4] 2> 308 T1154 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@12a91bf name:ZooKeeperConnection Watcher:127.0.0.1:53275/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4:junit4] 2> 308 T1132 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4:junit4] 2> 309 T1137 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d329a25f00003 type:create cxid:0x2 zxid:0x1a txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4] 2> 310 T1132 oascc.SolrZkClient.makePath makePath: /live_nodes
> [junit4:junit4] 2> 311 T1132 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:35276_ds_
> [junit4:junit4] 2> 312 T1137 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d329a25f00003 type:delete cxid:0x8 zxid:0x1c txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:35276_ds_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:35276_ds_
> [junit4:junit4] 2> 312 T1132 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:35276_ds_
> [junit4:junit4] 2> 314 T1132 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
> [junit4:junit4] 2> 316 T1137 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d329a25f00003 type:delete cxid:0x16 zxid:0x21 txntype:-1 reqpath:n/a Error Path:/solr/overseer_elect/leader Error:KeeperErrorCode = NoNode for /solr/overseer_elect/leader
> [junit4:junit4] 2> 317 T1132 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
> [junit4:junit4] 2> 319 T1132 oasc.Overseer.start Overseer (id=89283205260115971-127.0.0.1:35276_ds_-n_0000000000) starting
> [junit4:junit4] 2> 319 T1137 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d329a25f00003 type:create cxid:0x1b zxid:0x23 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4] 2> 320 T1137 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d329a25f00003 type:create cxid:0x1c zxid:0x24 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4] 2> 320 T1137 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d329a25f00003 type:create cxid:0x1d zxid:0x25 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4] 2> 321 T1137 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d329a25f00003 type:create cxid:0x1e zxid:0x26 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4] 2> 322 T1156 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
> [junit4:junit4] 2> 322 T1132 oascc.SolrZkClient.makePath makePath: /clusterstate.json
> [junit4:junit4] 2> 324 T1132 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
> [junit4:junit4] 2> 325 T1155 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
> [junit4:junit4] 2> 327 T1157 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1362353596046/collection1
> [junit4:junit4] 2> 328 T1157 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
> [junit4:junit4] 2> 328 T1157 oasc.ZkController.createCollectionZkNode Collection zkNode exists
> [junit4:junit4] 2> 328 T1157 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
> [junit4:junit4] 2> 329 T1157 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1362353596046/collection1/'
> [junit4:junit4] 2> 329 T1157 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1362353596046/collection1/lib/README' to classloader
> [junit4:junit4] 2> 330 T1157 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1362353596046/collection1/lib/classes/' to classloader
> [junit4:junit4] 2> 355 T1157 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
> [junit4:junit4] 2> 395 T1157 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
> [junit4:junit4] 2> 396 T1157 oass.IndexSchema.readSchema Reading Solr Schema
> [junit4:junit4] 2> 400 T1157 oass.IndexSchema.readSchema Schema name=test
> [junit4:junit4] 2> 802 T1157 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
> [junit4:junit4] 2> 807 T1157 oass.IndexSchema.readSchema default search field in schema is text
> [junit4:junit4] 2> 810 T1157 oass.IndexSchema.readSchema unique key field: id
> [junit4:junit4] 2> 819 T1157 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
> [junit4:junit4] 2> 822 T1157 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
> [junit4:junit4] 2> 825 T1157 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
> [junit4:junit4] 2> 826 T1157 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
> [junit4:junit4] 2> 826 T1157 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
> [junit4:junit4] 2> 827 T1157 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
> [junit4:junit4] 2> 828 T1157 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
> [junit4:junit4] 2> 828 T1157 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
> [junit4:junit4] 2> 828 T1157 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1362353596046/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/control/data/
> [junit4:junit4] 2> 828 T1157 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@64b87b
> [junit4:junit4] 2> 828 T1157 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
> [junit4:junit4] 2> 829 T1157 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/control/data forceNew:false
> [junit4:junit4] 2> 829 T1157 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/control/data
> [junit4:junit4] 2> 829 T1157 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/control/data/index/
> [junit4:junit4] 2> 830 T1157 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/control/data/index' doesn't exist. Creating new index...
> [junit4:junit4] 2> 830 T1157 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/control/data/index forceNew:false
> [junit4:junit4] 2> 831 T1157 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
> [junit4:junit4] 2> commit{dir=MockDirWrapper(NRTCachingDirectory(org.apache.lucene.store.RAMDirectory@53381c lockFactory=org.apache.lucene.store.NativeFSLockFactory@d4aa65; maxCacheMB=0.4580078125 maxMergeSizeMB=0.19140625)),segFN=segments_1,generation=1,filenames=[segments_1]
> [junit4:junit4] 2> 831 T1157 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
> [junit4:junit4] 2> 832 T1157 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/control/data/index
> [junit4:junit4] 2> 832 T1157 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
> [junit4:junit4] 2> 833 T1157 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
> [junit4:junit4] 2> 833 T1157 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
> [junit4:junit4] 2> 833 T1157 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
> [junit4:junit4] 2> 834 T1157 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
> [junit4:junit4] 2> 834 T1157 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
> [junit4:junit4] 2> 834 T1157 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
> [junit4:junit4] 2> 834 T1157 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
> [junit4:junit4] 2> 835 T1157 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
> [junit4:junit4] 2> 837 T1157 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
> [junit4:junit4] 2> 839 T1157 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/control/data
> [junit4:junit4] 2> 839 T1157 oass.SolrIndexSearcher.<init> Opening Searcher@47b90d main
> [junit4:junit4] 2> 840 T1157 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/control/data/tlog
> [junit4:junit4] 2> 840 T1157 oasu.CommitTracker.<init> Hard AutoCommit: disabled
> [junit4:junit4] 2> 840 T1157 oasu.CommitTracker.<init> Soft AutoCommit: disabled
> [junit4:junit4] 2> 842 T1157 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/control/data
> [junit4:junit4] 2> 843 T1158 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@47b90d main{StandardDirectoryReader(segments_1:1)}
> [junit4:junit4] 2> 843 T1157 oasc.ZkController.publish publishing core=collection1 state=down
> [junit4:junit4] 2> 844 T1157 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
> [junit4:junit4] 2> 844 T1137 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d329a25f00003 type:create cxid:0x50 zxid:0x29 txntype:-1 reqpath:n/a Error Path:/solr/overseer/queue Error:KeeperErrorCode = NoNode for /solr/overseer/queue
> [junit4:junit4] 2> 1829 T1155 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 1830 T1155 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
> [junit4:junit4] 2> "roles":null,
> [junit4:junit4] 2> "operation":"state",
> [junit4:junit4] 2> "collection":"control_collection",
> [junit4:junit4] 2> "shard":null,
> [junit4:junit4] 2> "core":"collection1",
> [junit4:junit4] 2> "base_url":"http://127.0.0.1:35276/ds_",
> [junit4:junit4] 2> "state":"down",
> [junit4:junit4] 2> "core_node_name":"1",
> [junit4:junit4] 2> "numShards":"1",
> [junit4:junit4] 2> "node_name":"127.0.0.1:35276_ds_"}
> [junit4:junit4] 2> 1830 T1155 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
> [junit4:junit4] 2> 1830 T1155 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
> [junit4:junit4] 2> 1831 T1137 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d329a25f00003 type:create cxid:0x58 zxid:0x2c txntype:-1 reqpath:n/a Error Path:/solr/overseer/queue-work Error:KeeperErrorCode = NoNode for /solr/overseer/queue-work
> [junit4:junit4] 2> 1834 T1154 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
> [junit4:junit4] 2> 1846 T1157 oasc.CoreContainer.registerCore registering core: collection1
> [junit4:junit4] 2> 1846 T1157 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:35276/ds_ collection:control_collection shard:shard1
> [junit4:junit4] 2> 1847 T1157 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
> [junit4:junit4] 2> 1853 T1137 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d329a25f00003 type:delete cxid:0x72 zxid:0x36 txntype:-1 reqpath:n/a Error Path:/solr/collections/control_collection/leaders Error:KeeperErrorCode = NoNode for /solr/collections/control_collection/leaders
> [junit4:junit4] 2> 1853 T1157 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
> [junit4:junit4] 2> 1854 T1137 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d329a25f00003 type:create cxid:0x73 zxid:0x37 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4] 2> 1855 T1157 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
> [junit4:junit4] 2> 1855 T1157 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
> [junit4:junit4] 2> 1856 T1157 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:35276/ds_/collection1/
> [junit4:junit4] 2> 1856 T1157 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
> [junit4:junit4] 2> 1856 T1157 oasc.SyncStrategy.syncToMe http://127.0.0.1:35276/ds_/collection1/ has no replicas
> [junit4:junit4] 2> 1856 T1157 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:35276/ds_/collection1/
> [junit4:junit4] 2> 1856 T1157 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
> [junit4:junit4] 2> 1859 T1137 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d329a25f00003 type:create cxid:0x7d zxid:0x3b txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4] 2> 3338 T1155 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 3344 T1154 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
> [junit4:junit4] 2> 3364 T1157 oasc.ZkController.register We are http://127.0.0.1:35276/ds_/collection1/ and leader is http://127.0.0.1:35276/ds_/collection1/
> [junit4:junit4] 2> 3364 T1157 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:35276/ds_
> [junit4:junit4] 2> 3364 T1157 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
> [junit4:junit4] 2> 3365 T1157 oasc.ZkController.publish publishing core=collection1 state=active
> [junit4:junit4] 2> 3365 T1157 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
> [junit4:junit4] 2> 3366 T1157 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 3367 T1132 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
> [junit4:junit4] 2> 3367 T1132 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
> [junit4:junit4] 2> 3368 T1132 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
> [junit4:junit4] 2> 3371 T1132 oascsi.HttpClientUtil.createClient Creating new http client, config:
> [junit4:junit4] 2> 3372 T1132 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:53275/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@2187f3
> [junit4:junit4] 2> 3373 T1132 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4:junit4] 2> 3373 T1160 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:53275. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
> [junit4:junit4] 2> 3373 T1160 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:53275, initiating session
> [junit4:junit4] 2> 3373 T1134 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:43974
> [junit4:junit4] 2> 3374 T1134 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:43974
> [junit4:junit4] 2> 3374 T1136 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d329a25f00004 with negotiated timeout 10000 for client /127.0.0.1:43974
> [junit4:junit4] 2> 3374 T1160 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:53275, sessionid = 0x13d329a25f00004, negotiated timeout = 10000
> [junit4:junit4] 2> 3375 T1161 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2187f3 name:ZooKeeperConnection Watcher:127.0.0.1:53275/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4:junit4] 2> 3375 T1132 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4:junit4] 2> 3376 T1132 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
> [junit4:junit4] 2> 3377 T1132 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
> [junit4:junit4] 2> 3438 T1132 oejs.Server.doStart jetty-8.1.8.v20121106
> [junit4:junit4] 2> 3440 T1132 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:58158
> [junit4:junit4] 2> 3440 T1132 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
> [junit4:junit4] 2> 3441 T1132 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
> [junit4:junit4] 2> 3441 T1132 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1362353599259
> [junit4:junit4] 2> 3441 T1132 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1362353599259/solr.xml
> [junit4:junit4] 2> 3442 T1132 oasc.CoreContainer.<init> New CoreContainer 7335748
> [junit4:junit4] 2> 3442 T1132 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1362353599259/'
> [junit4:junit4] 2> 3442 T1132 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1362353599259/'
> [junit4:junit4] 2> 3461 T1132 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
> [junit4:junit4] 2> 3461 T1132 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
> [junit4:junit4] 2> 3461 T1132 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
> [junit4:junit4] 2> 3462 T1132 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
> [junit4:junit4] 2> 3462 T1132 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
> [junit4:junit4] 2> 3462 T1132 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
> [junit4:junit4] 2> 3463 T1132 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
> [junit4:junit4] 2> 3463 T1132 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
> [junit4:junit4] 2> 3463 T1132 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
> [junit4:junit4] 2> 3464 T1132 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
> [junit4:junit4] 2> 3471 T1132 oasc.CoreContainer.load Registering Log Listener
> [junit4:junit4] 2> 3490 T1132 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:53275/solr
> [junit4:junit4] 2> 3490 T1132 oasc.ZkController.checkChrootPath zkHost includes chroot
> [junit4:junit4] 2> 3491 T1132 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:53275 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@1345f8c
> [junit4:junit4] 2> 3492 T1132 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4:junit4] 2> 3492 T1171 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:53275. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
> [junit4:junit4] 2> 3492 T1171 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:53275, initiating session
> [junit4:junit4] 2> 3493 T1134 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:43975
> [junit4:junit4] 2> 3493 T1134 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:43975
> [junit4:junit4] 2> 3494 T1136 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d329a25f00005 with negotiated timeout 20000 for client /127.0.0.1:43975
> [junit4:junit4] 2> 3494 T1171 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:53275, sessionid = 0x13d329a25f00005, negotiated timeout = 20000
> [junit4:junit4] 2> 3494 T1172 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1345f8c name:ZooKeeperConnection Watcher:127.0.0.1:53275 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4:junit4] 2> 3494 T1132 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4:junit4] 2> 3495 T1137 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d329a25f00005
> [junit4:junit4] 2> 3496 T1134 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:43975 which had sessionid 0x13d329a25f00005
> [junit4:junit4] 2> 3495 T1132 oaz.ZooKeeper.close Session: 0x13d329a25f00005 closed
> [junit4:junit4] 2> 3496 T1132 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
> [junit4:junit4] 2> 3497 T1172 oaz.ClientCnxn$EventThread.run EventThread shut down
> [junit4:junit4] 2> 3499 T1132 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:53275/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@c336c0
> [junit4:junit4] 2> 3500 T1173 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:53275. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
> [junit4:junit4] 2> 3499 T1132 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4:junit4] 2> 3500 T1173 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:53275, initiating session
> [junit4:junit4] 2> 3500 T1134 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:43976
> [junit4:junit4] 2> 3500 T1134 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:43976
> [junit4:junit4] 2> 3501 T1136 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d329a25f00006 with negotiated timeout 20000 for client /127.0.0.1:43976
> [junit4:junit4] 2> 3501 T1173 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:53275, sessionid = 0x13d329a25f00006, negotiated timeout = 20000
> [junit4:junit4] 2> 3501 T1174 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@c336c0 name:ZooKeeperConnection Watcher:127.0.0.1:53275/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4:junit4] 2> 3502 T1132 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4:junit4] 2> 3502 T1137 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d329a25f00006 type:create cxid:0x1 zxid:0x49 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4] 2> 3503 T1137 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d329a25f00006 type:create cxid:0x2 zxid:0x4a txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4] 2> 3504 T1132 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
> [junit4:junit4] 2> 4506 T1132 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:58158_ds_
> [junit4:junit4] 2> 4507 T1137 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d329a25f00006 type:delete cxid:0x9 zxid:0x4b txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:58158_ds_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:58158_ds_
> [junit4:junit4] 2> 4507 T1132 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:58158_ds_
> [junit4:junit4] 2> 4510 T1174 oascc.ZkStateReader$3.process Updating live nodes... (2)
> [junit4:junit4] 2> 4510 T1154 oascc.ZkStateReader$3.process Updating live nodes... (2)
> [junit4:junit4] 2> 4511 T1154 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
> [junit4:junit4] 2> 4511 T1161 oascc.ZkStateReader$3.process Updating live nodes... (2)
> [junit4:junit4] 2> 4515 T1175 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1362353599259/collection1
> [junit4:junit4] 2> 4515 T1175 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
> [junit4:junit4] 2> 4516 T1175 oasc.ZkController.createCollectionZkNode Collection zkNode exists
> [junit4:junit4] 2> 4516 T1175 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
> [junit4:junit4] 2> 4517 T1175 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1362353599259/collection1/'
> [junit4:junit4] 2> 4517 T1175 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1362353599259/collection1/lib/README' to classloader
> [junit4:junit4] 2> 4518 T1175 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1362353599259/collection1/lib/classes/' to classloader
> [junit4:junit4] 2> 4543 T1175 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
> [junit4:junit4] 2> 4584 T1175 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
> [junit4:junit4] 2> 4585 T1175 oass.IndexSchema.readSchema Reading Solr Schema
> [junit4:junit4] 2> 4589 T1175 oass.IndexSchema.readSchema Schema name=test
> [junit4:junit4] 2> 4848 T1155 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 4849 T1155 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
> [junit4:junit4] 2> "roles":null,
> [junit4:junit4] 2> "operation":"state",
> [junit4:junit4] 2> "collection":"control_collection",
> [junit4:junit4] 2> "shard":"shard1",
> [junit4:junit4] 2> "core":"collection1",
> [junit4:junit4] 2> "base_url":"http://127.0.0.1:35276/ds_",
> [junit4:junit4] 2> "state":"active",
> [junit4:junit4] 2> "core_node_name":"1",
> [junit4:junit4] 2> "numShards":"1",
> [junit4:junit4] 2> "node_name":"127.0.0.1:35276_ds_"}
> [junit4:junit4] 2> 4851 T1154 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
> [junit4:junit4] 2> 4852 T1161 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
> [junit4:junit4] 2> 4852 T1174 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
> [junit4:junit4] 2> 4995 T1175 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
> [junit4:junit4] 2> 5001 T1175 oass.IndexSchema.readSchema default search field in schema is text
> [junit4:junit4] 2> 5004 T1175 oass.IndexSchema.readSchema unique key field: id
> [junit4:junit4] 2> 5012 T1175 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
> [junit4:junit4] 2> 5015 T1175 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
> [junit4:junit4] 2> 5017 T1175 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
> [junit4:junit4] 2> 5018 T1175 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
> [junit4:junit4] 2> 5018 T1175 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
> [junit4:junit4] 2> 5018 T1175 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
> [junit4:junit4] 2> 5019 T1175 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
> [junit4:junit4] 2> 5019 T1175 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
> [junit4:junit4] 2> 5020 T1175 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1362353599259/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty1/
> [junit4:junit4] 2> 5020 T1175 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@64b87b
> [junit4:junit4] 2> 5020 T1175 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
> [junit4:junit4] 2> 5020 T1175 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty1 forceNew:false
> [junit4:junit4] 2> 5021 T1175 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty1
> [junit4:junit4] 2> 5021 T1175 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty1/index/
> [junit4:junit4] 2> 5021 T1175 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty1/index' doesn't exist. Creating new index...
> [junit4:junit4] 2> 5021 T1175 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty1/index forceNew:false
> [junit4:junit4] 2> 5022 T1175 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
> [junit4:junit4] 2> commit{dir=MockDirWrapper(NRTCachingDirectory(org.apache.lucene.store.RAMDirectory@46ba6b lockFactory=org.apache.lucene.store.NativeFSLockFactory@227076; maxCacheMB=0.4580078125 maxMergeSizeMB=0.19140625)),segFN=segments_1,generation=1,filenames=[segments_1]
> [junit4:junit4] 2> 5023 T1175 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
> [junit4:junit4] 2> 5023 T1175 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty1/index
> [junit4:junit4] 2> 5024 T1175 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
> [junit4:junit4] 2> 5024 T1175 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
> [junit4:junit4] 2> 5025 T1175 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
> [junit4:junit4] 2> 5025 T1175 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
> [junit4:junit4] 2> 5026 T1175 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
> [junit4:junit4] 2> 5026 T1175 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
> [junit4:junit4] 2> 5026 T1175 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
> [junit4:junit4] 2> 5026 T1175 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
> [junit4:junit4] 2> 5027 T1175 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
> [junit4:junit4] 2> 5029 T1175 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
> [junit4:junit4] 2> 5031 T1175 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty1
> [junit4:junit4] 2> 5031 T1175 oass.SolrIndexSearcher.<init> Opening Searcher@465f19 main
> [junit4:junit4] 2> 5032 T1175 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty1/tlog
> [junit4:junit4] 2> 5032 T1175 oasu.CommitTracker.<init> Hard AutoCommit: disabled
> [junit4:junit4] 2> 5032 T1175 oasu.CommitTracker.<init> Soft AutoCommit: disabled
> [junit4:junit4] 2> 5036 T1176 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@465f19 main{StandardDirectoryReader(segments_1:1)}
> [junit4:junit4] 2> 5037 T1175 oasc.ZkController.publish publishing core=collection1 state=down
> [junit4:junit4] 2> 5037 T1175 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
> [junit4:junit4] 2> 6355 T1155 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 6356 T1155 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
> [junit4:junit4] 2> "roles":null,
> [junit4:junit4] 2> "operation":"state",
> [junit4:junit4] 2> "collection":"collection1",
> [junit4:junit4] 2> "shard":null,
> [junit4:junit4] 2> "core":"collection1",
> [junit4:junit4] 2> "base_url":"http://127.0.0.1:58158/ds_",
> [junit4:junit4] 2> "state":"down",
> [junit4:junit4] 2> "core_node_name":"2",
> [junit4:junit4] 2> "numShards":"2",
> [junit4:junit4] 2> "node_name":"127.0.0.1:58158_ds_"}
> [junit4:junit4] 2> 6356 T1155 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 2
> [junit4:junit4] 2> 6356 T1155 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
> [junit4:junit4] 2> 6360 T1161 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
> [junit4:junit4] 2> 6360 T1154 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
> [junit4:junit4] 2> 6360 T1174 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
> [junit4:junit4] 2> 7039 T1175 oasc.CoreContainer.registerCore registering core: collection1
> [junit4:junit4] 2> 7039 T1175 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:58158/ds_ collection:collection1 shard:shard2
> [junit4:junit4] 2> 7040 T1175 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
> [junit4:junit4] 2> 7044 T1137 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d329a25f00006 type:delete cxid:0x46 zxid:0x5b txntype:-1 reqpath:n/a Error Path:/solr/collections/collection1/leaders Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders
> [junit4:junit4] 2> 7045 T1175 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
> [junit4:junit4] 2> 7045 T1137 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d329a25f00006 type:create cxid:0x47 zxid:0x5c txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4] 2> 7046 T1175 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
> [junit4:junit4] 2> 7047 T1175 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
> [junit4:junit4] 2> 7047 T1175 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:58158/ds_/collection1/
> [junit4:junit4] 2> 7047 T1175 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
> [junit4:junit4] 2> 7047 T1175 oasc.SyncStrategy.syncToMe http://127.0.0.1:58158/ds_/collection1/ has no replicas
> [junit4:junit4] 2> 7047 T1175 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:58158/ds_/collection1/
> [junit4:junit4] 2> 7047 T1175 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
> [junit4:junit4] 2> 7050 T1137 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d329a25f00006 type:create cxid:0x51 zxid:0x60 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4] 2> 7863 T1155 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 7869 T1161 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
> [junit4:junit4] 2> 7869 T1174 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
> [junit4:junit4] 2> 7869 T1154 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
> [junit4:junit4] 2> 7903 T1175 oasc.ZkController.register We are http://127.0.0.1:58158/ds_/collection1/ and leader is http://127.0.0.1:58158/ds_/collection1/
> [junit4:junit4] 2> 7903 T1175 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:58158/ds_
> [junit4:junit4] 2> 7903 T1175 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
> [junit4:junit4] 2> 7904 T1175 oasc.ZkController.publish publishing core=collection1 state=active
> [junit4:junit4] 2> 7904 T1175 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
> [junit4:junit4] 2> 7905 T1175 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 7906 T1132 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
> [junit4:junit4] 2> 7907 T1132 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
> [junit4:junit4] 2> 7907 T1132 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
> [junit4:junit4] 2> 7970 T1132 oejs.Server.doStart jetty-8.1.8.v20121106
> [junit4:junit4] 2> 7972 T1132 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:49875
> [junit4:junit4] 2> 7972 T1132 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
> [junit4:junit4] 2> 7973 T1132 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
> [junit4:junit4] 2> 7973 T1132 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1362353603792
> [junit4:junit4] 2> 7974 T1132 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1362353603792/solr.xml
> [junit4:junit4] 2> 7974 T1132 oasc.CoreContainer.<init> New CoreContainer 9096842
> [junit4:junit4] 2> 7974 T1132 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1362353603792/'
> [junit4:junit4] 2> 7975 T1132 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1362353603792/'
> [junit4:junit4] 2> 7999 T1132 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
> [junit4:junit4] 2> 8000 T1132 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
> [junit4:junit4] 2> 8000 T1132 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
> [junit4:junit4] 2> 8000 T1132 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
> [junit4:junit4] 2> 8001 T1132 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
> [junit4:junit4] 2> 8001 T1132 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
> [junit4:junit4] 2> 8001 T1132 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
> [junit4:junit4] 2> 8002 T1132 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
> [junit4:junit4] 2> 8002 T1132 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
> [junit4:junit4] 2> 8002 T1132 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
> [junit4:junit4] 2> 8009 T1132 oasc.CoreContainer.load Registering Log Listener
> [junit4:junit4] 2> 8021 T1132 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:53275/solr
> [junit4:junit4] 2> 8022 T1132 oasc.ZkController.checkChrootPath zkHost includes chroot
> [junit4:junit4] 2> 8022 T1132 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:53275 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@91fade
> [junit4:junit4] 2> 8023 T1132 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4:junit4] 2> 8023 T1187 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:53275. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
> [junit4:junit4] 2> 8024 T1187 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:53275, initiating session
> [junit4:junit4] 2> 8024 T1134 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:43979
> [junit4:junit4] 2> 8024 T1134 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:43979
> [junit4:junit4] 2> 8025 T1136 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d329a25f00007 with negotiated timeout 20000 for client /127.0.0.1:43979
> [junit4:junit4] 2> 8025 T1187 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:53275, sessionid = 0x13d329a25f00007, negotiated timeout = 20000
> [junit4:junit4] 2> 8025 T1188 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@91fade name:ZooKeeperConnection Watcher:127.0.0.1:53275 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4:junit4] 2> 8026 T1132 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4:junit4] 2> 8027 T1137 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d329a25f00007
> [junit4:junit4] 2> 8027 T1188 oaz.ClientCnxn$EventThread.run EventThread shut down
> [junit4:junit4] 2> 8027 T1134 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:43979 which had sessionid 0x13d329a25f00007
> [junit4:junit4] 2> 8027 T1132 oaz.ZooKeeper.close Session: 0x13d329a25f00007 closed
> [junit4:junit4] 2> 8028 T1132 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
> [junit4:junit4] 2> 8030 T1132 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:53275/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@10122f8
> [junit4:junit4] 2> 8031 T1189 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:53275. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
> [junit4:junit4] 2> 8031 T1132 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4:junit4] 2> 8031 T1189 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:53275, initiating session
> [junit4:junit4] 2> 8031 T1134 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:43980
> [junit4:junit4] 2> 8032 T1134 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:43980
> [junit4:junit4] 2> 8032 T1136 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d329a25f00008 with negotiated timeout 20000 for client /127.0.0.1:43980
> [junit4:junit4] 2> 8032 T1189 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:53275, sessionid = 0x13d329a25f00008, negotiated timeout = 20000
> [junit4:junit4] 2> 8033 T1190 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@10122f8 name:ZooKeeperConnection Watcher:127.0.0.1:53275/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4:junit4] 2> 8033 T1132 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4:junit4] 2> 8034 T1137 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d329a25f00008 type:create cxid:0x1 zxid:0x6d txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4] 2> 8034 T1137 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d329a25f00008 type:create cxid:0x2 zxid:0x6e txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4] 2> 8035 T1132 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
> [junit4:junit4] 2> 9037 T1132 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:49875_ds_
> [junit4:junit4] 2> 9038 T1137 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d329a25f00008 type:delete cxid:0x9 zxid:0x6f txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:49875_ds_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:49875_ds_
> [junit4:junit4] 2> 9038 T1132 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:49875_ds_
> [junit4:junit4] 2> 9040 T1161 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
> [junit4:junit4] 2> 9041 T1154 oascc.ZkStateReader$3.process Updating live nodes... (3)
> [junit4:junit4] 2> 9041 T1174 oascc.ZkStateReader$3.process Updating live nodes... (3)
> [junit4:junit4] 2> 9041 T1154 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
> [junit4:junit4] 2> 9041 T1190 oascc.ZkStateReader$3.process Updating live nodes... (3)
> [junit4:junit4] 2> 9041 T1174 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
> [junit4:junit4] 2> 9042 T1161 oascc.ZkStateReader$3.process Updating live nodes... (3)
> [junit4:junit4] 2> 9046 T1191 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1362353603792/collection1
> [junit4:junit4] 2> 9046 T1191 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
> [junit4:junit4] 2> 9046 T1191 oasc.ZkController.createCollectionZkNode Collection zkNode exists
> [junit4:junit4] 2> 9047 T1191 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
> [junit4:junit4] 2> 9048 T1191 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1362353603792/collection1/'
> [junit4:junit4] 2> 9048 T1191 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1362353603792/collection1/lib/README' to classloader
> [junit4:junit4] 2> 9048 T1191 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1362353603792/collection1/lib/classes/' to classloader
> [junit4:junit4] 2> 9074 T1191 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
> [junit4:junit4] 2> 9116 T1191 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
> [junit4:junit4] 2> 9117 T1191 oass.IndexSchema.readSchema Reading Solr Schema
> [junit4:junit4] 2> 9121 T1191 oass.IndexSchema.readSchema Schema name=test
> [junit4:junit4] 2> 9373 T1155 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 9374 T1155 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
> [junit4:junit4] 2> "roles":null,
> [junit4:junit4] 2> "operation":"state",
> [junit4:junit4] 2> "collection":"collection1",
> [junit4:junit4] 2> "shard":"shard2",
> [junit4:junit4] 2> "core":"collection1",
> [junit4:junit4] 2> "base_url":"http://127.0.0.1:58158/ds_",
> [junit4:junit4] 2> "state":"active",
> [junit4:junit4] 2> "core_node_name":"2",
> [junit4:junit4] 2> "numShards":"2",
> [junit4:junit4] 2> "node_name":"127.0.0.1:58158_ds_"}
> [junit4:junit4] 2> 9377 T1154 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4:junit4] 2> 9386 T1190 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4:junit4] 2> 9386 T1161 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4:junit4] 2> 9386 T1174 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4:junit4] 2> 9574 T1191 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
> [junit4:junit4] 2> 9580 T1191 oass.IndexSchema.readSchema default search field in schema is text
> [junit4:junit4] 2> 9582 T1191 oass.IndexSchema.readSchema unique key field: id
> [junit4:junit4] 2> 9594 T1191 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
> [junit4:junit4] 2> 9597 T1191 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
> [junit4:junit4] 2> 9600 T1191 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
> [junit4:junit4] 2> 9600 T1191 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
> [junit4:junit4] 2> 9601 T1191 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
> [junit4:junit4] 2> 9601 T1191 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
> [junit4:junit4] 2> 9602 T1191 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
> [junit4:junit4] 2> 9602 T1191 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
> [junit4:junit4] 2> 9602 T1191 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1362353603792/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty2/
> [junit4:junit4] 2> 9603 T1191 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@64b87b
> [junit4:junit4] 2> 9603 T1191 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
> [junit4:junit4] 2> 9603 T1191 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty2 forceNew:false
> [junit4:junit4] 2> 9604 T1191 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty2
> [junit4:junit4] 2> 9604 T1191 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty2/index/
> [junit4:junit4] 2> 9604 T1191 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty2/index' doesn't exist. Creating new index...
> [junit4:junit4] 2> 9604 T1191 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty2/index forceNew:false
> [junit4:junit4] 2> 9605 T1191 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
> [junit4:junit4] 2> commit{dir=MockDirWrapper(NRTCachingDirectory(org.apache.lucene.store.RAMDirectory@fcbcdc lockFactory=org.apache.lucene.store.NativeFSLockFactory@1bcbfb5; maxCacheMB=0.4580078125 maxMergeSizeMB=0.19140625)),segFN=segments_1,generation=1,filenames=[segments_1]
> [junit4:junit4] 2> 9606 T1191 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
> [junit4:junit4] 2> 9606 T1191 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty2/index
> [junit4:junit4] 2> 9607 T1191 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
> [junit4:junit4] 2> 9607 T1191 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
> [junit4:junit4] 2> 9608 T1191 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
> [junit4:junit4] 2> 9608 T1191 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
> [junit4:junit4] 2> 9608 T1191 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
> [junit4:junit4] 2> 9609 T1191 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
> [junit4:junit4] 2> 9609 T1191 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
> [junit4:junit4] 2> 9609 T1191 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
> [junit4:junit4] 2> 9610 T1191 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
> [junit4:junit4] 2> 9612 T1191 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
> [junit4:junit4] 2> 9614 T1191 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty2
> [junit4:junit4] 2> 9615 T1191 oass.SolrIndexSearcher.<init> Opening Searcher@1735c9e main
> [junit4:junit4] 2> 9615 T1191 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty2/tlog
> [junit4:junit4] 2> 9615 T1191 oasu.CommitTracker.<init> Hard AutoCommit: disabled
> [junit4:junit4] 2> 9616 T1191 oasu.CommitTracker.<init> Soft AutoCommit: disabled
> [junit4:junit4] 2> 9619 T1192 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1735c9e main{StandardDirectoryReader(segments_1:1)}
> [junit4:junit4] 2> 9621 T1191 oasc.ZkController.publish publishing core=collection1 state=down
> [junit4:junit4] 2> 9621 T1191 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
> [junit4:junit4] 2> 10890 T1155 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 10891 T1155 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
> [junit4:junit4] 2> "roles":null,
> [junit4:junit4] 2> "operation":"state",
> [junit4:junit4] 2> "collection":"collection1",
> [junit4:junit4] 2> "shard":null,
> [junit4:junit4] 2> "core":"collection1",
> [junit4:junit4] 2> "base_url":"http://127.0.0.1:49875/ds_",
> [junit4:junit4] 2> "state":"down",
> [junit4:junit4] 2> "core_node_name":"3",
> [junit4:junit4] 2> "numShards":"2",
> [junit4:junit4] 2> "node_name":"127.0.0.1:49875_ds_"}
> [junit4:junit4] 2> 10891 T1155 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
> [junit4:junit4] 2> 10891 T1155 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
> [junit4:junit4] 2> 10895 T1190 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4:junit4] 2> 10895 T1154 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4:junit4] 2> 10895 T1161 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4:junit4] 2> 10895 T1174 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4:junit4] 2> 11622 T1191 oasc.CoreContainer.registerCore registering core: collection1
> [junit4:junit4] 2> 11622 T1191 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:49875/ds_ collection:collection1 shard:shard1
> [junit4:junit4] 2> 11623 T1191 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
> [junit4:junit4] 2> 11627 T1137 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d329a25f00008 type:delete cxid:0x45 zxid:0x7e txntype:-1 reqpath:n/a Error Path:/solr/collections/collection1/leaders/shard1 Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders/shard1
> [junit4:junit4] 2> 11628 T1191 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
> [junit4:junit4] 2> 11628 T1137 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d329a25f00008 type:create cxid:0x46 zxid:0x7f txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4] 2> 11630 T1191 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
> [junit4:junit4] 2> 11630 T1191 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
> [junit4:junit4] 2> 11630 T1191 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:49875/ds_/collection1/
> [junit4:junit4] 2> 11630 T1191 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
> [junit4:junit4] 2> 11630 T1191 oasc.SyncStrategy.syncToMe http://127.0.0.1:49875/ds_/collection1/ has no replicas
> [junit4:junit4] 2> 11631 T1191 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:49875/ds_/collection1/
> [junit4:junit4] 2> 11631 T1191 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
> [junit4:junit4] 2> 11633 T1137 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d329a25f00008 type:create cxid:0x4f zxid:0x82 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4] 2> 12399 T1155 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 12406 T1190 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4:junit4] 2> 12406 T1161 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4:junit4] 2> 12406 T1154 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4:junit4] 2> 12406 T1174 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4:junit4] 2> 12436 T1191 oasc.ZkController.register We are http://127.0.0.1:49875/ds_/collection1/ and leader is http://127.0.0.1:49875/ds_/collection1/
> [junit4:junit4] 2> 12437 T1191 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:49875/ds_
> [junit4:junit4] 2> 12437 T1191 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
> [junit4:junit4] 2> 12437 T1191 oasc.ZkController.publish publishing core=collection1 state=active
> [junit4:junit4] 2> 12437 T1191 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
> [junit4:junit4] 2> 12439 T1191 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 12440 T1132 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
> [junit4:junit4] 2> 12440 T1132 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
> [junit4:junit4] 2> 12441 T1132 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
> [junit4:junit4] 2> 12504 T1132 oejs.Server.doStart jetty-8.1.8.v20121106
> [junit4:junit4] 2> 12513 T1132 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:59775
> [junit4:junit4] 2> 12514 T1132 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
> [junit4:junit4] 2> 12514 T1132 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
> [junit4:junit4] 2> 12515 T1132 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1362353608325
> [junit4:junit4] 2> 12515 T1132 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1362353608325/solr.xml
> [junit4:junit4] 2> 12515 T1132 oasc.CoreContainer.<init> New CoreContainer 3438750
> [junit4:junit4] 2> 12516 T1132 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1362353608325/'
> [junit4:junit4] 2> 12516 T1132 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1362353608325/'
> [junit4:junit4] 2> 12533 T1132 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
> [junit4:junit4] 2> 12534 T1132 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
> [junit4:junit4] 2> 12534 T1132 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
> [junit4:junit4] 2> 12534 T1132 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
> [junit4:junit4] 2> 12535 T1132 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
> [junit4:junit4] 2> 12535 T1132 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
> [junit4:junit4] 2> 12535 T1132 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
> [junit4:junit4] 2> 12536 T1132 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
> [junit4:junit4] 2> 12536 T1132 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
> [junit4:junit4] 2> 12536 T1132 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
> [junit4:junit4] 2> 12543 T1132 oasc.CoreContainer.load Registering Log Listener
> [junit4:junit4] 2> 12554 T1132 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:53275/solr
> [junit4:junit4] 2> 12554 T1132 oasc.ZkController.checkChrootPath zkHost includes chroot
> [junit4:junit4] 2> 12555 T1132 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:53275 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@5762a9
> [junit4:junit4] 2> 12556 T1132 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4:junit4] 2> 12556 T1203 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:53275. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
> [junit4:junit4] 2> 12557 T1203 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:53275, initiating session
> [junit4:junit4] 2> 12557 T1134 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:43988
> [junit4:junit4] 2> 12557 T1134 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:43988
> [junit4:junit4] 2> 12558 T1136 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d329a25f00009 with negotiated timeout 20000 for client /127.0.0.1:43988
> [junit4:junit4] 2> 12558 T1203 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:53275, sessionid = 0x13d329a25f00009, negotiated timeout = 20000
> [junit4:junit4] 2> 12558 T1204 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5762a9 name:ZooKeeperConnection Watcher:127.0.0.1:53275 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4:junit4] 2> 12558 T1132 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4:junit4] 2> 12559 T1137 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d329a25f00009
> [junit4:junit4] 2> 12560 T1204 oaz.ClientCnxn$EventThread.run EventThread shut down
> [junit4:junit4] 2> 12560 T1134 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:43988 which had sessionid 0x13d329a25f00009
> [junit4:junit4] 2> 12560 T1132 oaz.ZooKeeper.close Session: 0x13d329a25f00009 closed
> [junit4:junit4] 2> 12560 T1132 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
> [junit4:junit4] 2> 12563 T1132 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:53275/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@52281e
> [junit4:junit4] 2> 12564 T1205 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:53275. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
> [junit4:junit4] 2> 12564 T1132 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4:junit4] 2> 12564 T1134 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:43989
> [junit4:junit4] 2> 12564 T1205 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:53275, initiating session
> [junit4:junit4] 2> 12565 T1134 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:43989
> [junit4:junit4] 2> 12565 T1136 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d329a25f0000a with negotiated timeout 20000 for client /127.0.0.1:43989
> [junit4:junit4] 2> 12565 T1205 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:53275, sessionid = 0x13d329a25f0000a, negotiated timeout = 20000
> [junit4:junit4] 2> 12566 T1206 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@52281e name:ZooKeeperConnection Watcher:127.0.0.1:53275/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4:junit4] 2> 12566 T1132 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4:junit4] 2> 12567 T1137 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d329a25f0000a type:create cxid:0x1 zxid:0x8f txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4] 2> 12567 T1137 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d329a25f0000a type:create cxid:0x2 zxid:0x90 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4] 2> 12568 T1132 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
> [junit4:junit4] 2> 13571 T1132 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:59775_ds_
> [junit4:junit4] 2> 13571 T1137 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d329a25f0000a type:delete cxid:0x9 zxid:0x91 txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:59775_ds_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:59775_ds_
> [junit4:junit4] 2> 13572 T1132 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:59775_ds_
> [junit4:junit4] 2> 13574 T1174 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
> [junit4:junit4] 2> 13575 T1154 oascc.ZkStateReader$3.process Updating live nodes... (4)
> [junit4:junit4] 2> 13575 T1190 oascc.ZkStateReader$3.process Updating live nodes... (4)
> [junit4:junit4] 2> 13575 T1161 oascc.ZkStateReader$3.process Updating live nodes... (4)
> [junit4:junit4] 2> 13575 T1154 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 13575 T1190 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 13576 T1161 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 13576 T1206 oascc.ZkStateReader$3.process Updating live nodes... (4)
> [junit4:junit4] 2> 13577 T1174 oascc.ZkStateReader$3.process Updating live nodes... (4)
> [junit4:junit4] 2> 13582 T1207 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1362353608325/collection1
> [junit4:junit4] 2> 13582 T1207 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
> [junit4:junit4] 2> 13583 T1207 oasc.ZkController.createCollectionZkNode Collection zkNode exists
> [junit4:junit4] 2> 13583 T1207 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
> [junit4:junit4] 2> 13584 T1207 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1362353608325/collection1/'
> [junit4:junit4] 2> 13584 T1207 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1362353608325/collection1/lib/README' to classloader
> [junit4:junit4] 2> 13584 T1207 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1362353608325/collection1/lib/classes/' to classloader
> [junit4:junit4] 2> 13610 T1207 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
> [junit4:junit4] 2> 13651 T1207 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
> [junit4:junit4] 2> 13652 T1207 oass.IndexSchema.readSchema Reading Solr Schema
> [junit4:junit4] 2> 13656 T1207 oass.IndexSchema.readSchema Schema name=test
> [junit4:junit4] 2> 13911 T1155 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 13912 T1155 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
> [junit4:junit4] 2> "roles":null,
> [junit4:junit4] 2> "operation":"state",
> [junit4:junit4] 2> "collection":"collection1",
> [junit4:junit4] 2> "shard":"shard1",
> [junit4:junit4] 2> "core":"collection1",
> [junit4:junit4] 2> "base_url":"http://127.0.0.1:49875/ds_",
> [junit4:junit4] 2> "state":"active",
> [junit4:junit4] 2> "core_node_name":"3",
> [junit4:junit4] 2> "numShards":"2",
> [junit4:junit4] 2> "node_name":"127.0.0.1:49875_ds_"}
> [junit4:junit4] 2> 13914 T1154 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 13915 T1174 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 13915 T1161 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 13915 T1206 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 13915 T1190 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 14064 T1207 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
> [junit4:junit4] 2> 14070 T1207 oass.IndexSchema.readSchema default search field in schema is text
> [junit4:junit4] 2> 14073 T1207 oass.IndexSchema.readSchema unique key field: id
> [junit4:junit4] 2> 14082 T1207 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
> [junit4:junit4] 2> 14084 T1207 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
> [junit4:junit4] 2> 14087 T1207 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
> [junit4:junit4] 2> 14088 T1207 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
> [junit4:junit4] 2> 14088 T1207 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
> [junit4:junit4] 2> 14089 T1207 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
> [junit4:junit4] 2> 14089 T1207 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
> [junit4:junit4] 2> 14090 T1207 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
> [junit4:junit4] 2> 14090 T1207 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1362353608325/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty3/
> [junit4:junit4] 2> 14090 T1207 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@64b87b
> [junit4:junit4] 2> 14090 T1207 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
> [junit4:junit4] 2> 14091 T1207 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty3 forceNew:false
> [junit4:junit4] 2> 14091 T1207 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty3
> [junit4:junit4] 2> 14091 T1207 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty3/index/
> [junit4:junit4] 2> 14091 T1207 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty3/index' doesn't exist. Creating new index...
> [junit4:junit4] 2> 14092 T1207 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty3/index forceNew:false
> [junit4:junit4] 2> 14093 T1207 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
> [junit4:junit4] 2> commit{dir=MockDirWrapper(NRTCachingDirectory(org.apache.lucene.store.RAMDirectory@1b71845 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1bd6084; maxCacheMB=0.4580078125 maxMergeSizeMB=0.19140625)),segFN=segments_1,generation=1,filenames=[segments_1]
> [junit4:junit4] 2> 14093 T1207 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
> [junit4:junit4] 2> 14093 T1207 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty3/index
> [junit4:junit4] 2> 14094 T1207 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
> [junit4:junit4] 2> 14095 T1207 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
> [junit4:junit4] 2> 14095 T1207 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
> [junit4:junit4] 2> 14096 T1207 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
> [junit4:junit4] 2> 14096 T1207 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
> [junit4:junit4] 2> 14096 T1207 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
> [junit4:junit4] 2> 14096 T1207 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
> [junit4:junit4] 2> 14097 T1207 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
> [junit4:junit4] 2> 14097 T1207 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
> [junit4:junit4] 2> 14099 T1207 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
> [junit4:junit4] 2> 14102 T1207 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty3
> [junit4:junit4] 2> 14102 T1207 oass.SolrIndexSearcher.<init> Opening Searcher@1e91dfe main
> [junit4:junit4] 2> 14102 T1207 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty3/tlog
> [junit4:junit4] 2> 14103 T1207 oasu.CommitTracker.<init> Hard AutoCommit: disabled
> [junit4:junit4] 2> 14103 T1207 oasu.CommitTracker.<init> Soft AutoCommit: disabled
> [junit4:junit4] 2> 14108 T1208 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1e91dfe main{StandardDirectoryReader(segments_1:1)}
> [junit4:junit4] 2> 14109 T1207 oasc.ZkController.publish publishing core=collection1 state=down
> [junit4:junit4] 2> 14110 T1207 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
> [junit4:junit4] 2> 15419 T1155 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 15420 T1155 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
> [junit4:junit4] 2> "roles":null,
> [junit4:junit4] 2> "operation":"state",
> [junit4:junit4] 2> "collection":"collection1",
> [junit4:junit4] 2> "shard":null,
> [junit4:junit4] 2> "core":"collection1",
> [junit4:junit4] 2> "base_url":"http://127.0.0.1:59775/ds_",
> [junit4:junit4] 2> "state":"down",
> [junit4:junit4] 2> "core_node_name":"4",
> [junit4:junit4] 2> "numShards":"2",
> [junit4:junit4] 2> "node_name":"127.0.0.1:59775_ds_"}
> [junit4:junit4] 2> 15420 T1155 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
> [junit4:junit4] 2> 15420 T1155 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
> [junit4:junit4] 2> 15424 T1161 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 15424 T1154 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 15424 T1206 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 15424 T1174 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 15424 T1190 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 16111 T1207 oasc.CoreContainer.registerCore registering core: collection1
> [junit4:junit4] 2> 16111 T1207 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:59775/ds_ collection:collection1 shard:shard2
> [junit4:junit4] 2> 16114 T1207 oasc.ZkController.register We are http://127.0.0.1:59775/ds_/collection1/ and leader is http://127.0.0.1:58158/ds_/collection1/
> [junit4:junit4] 2> 16114 T1207 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:59775/ds_
> [junit4:junit4] 2> 16115 T1207 oasc.ZkController.checkRecovery Core needs to recover:collection1
> [junit4:junit4] 2> 16115 T1207 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
> [junit4:junit4] 2> ASYNC NEW_CORE C55 name=collection1 org.apache.solr.core.SolrCore@b9d6ee url=http://127.0.0.1:59775/ds_/collection1 node=127.0.0.1:59775_ds_ C55_STATE=coll:collection1 core:collection1 props:{collection=collection1, core=collection1, base_url=http://127.0.0.1:59775/ds_, state=down, node_name=127.0.0.1:59775_ds_}
> [junit4:junit4] 2> 16115 T1209 C55 P59775 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
> [junit4:junit4] 2> 16115 T1207 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 16116 T1209 C55 P59775 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
> [junit4:junit4] 2> 16116 T1209 C55 P59775 oasc.ZkController.publish publishing core=collection1 state=recovering
> [junit4:junit4] 2> 16116 T1209 C55 P59775 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
> [junit4:junit4] 2> 16117 T1132 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
> [junit4:junit4] 2> 16117 T1209 C55 P59775 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
> [junit4:junit4] 2> 16117 T1132 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
> [junit4:junit4] 2> 16118 T1132 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
> [junit4:junit4] 2> 16120 T1132 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 16121 T1132 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
> [junit4:junit4] 2> 16125 T1170 oasc.CoreContainer.create Creating SolrCore 'onenodecollectioncore' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1362353599259/onenodecollectioncore
> [junit4:junit4] 2> 16125 T1170 oasc.ZkController.createCollectionZkNode Check for collection zkNode:onenodecollection
> [junit4:junit4] 2> 16126 T1170 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:onenodecollection
> [junit4:junit4] 2> 16127 T1170 oasc.ZkController.getConfName Looking for collection configName
> [junit4:junit4] 2> 16128 T1170 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
> [junit4:junit4] 2> 16128 T1170 oascc.SolrZkClient.makePath makePath: /collections/onenodecollection
> [junit4:junit4] 2> 16130 T1170 oasc.ZkController.readConfigName Load collection config from:/collections/onenodecollection
> [junit4:junit4] 2> 16131 T1170 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1362353599259/onenodecollectioncore/'
> [junit4:junit4] 2> 16183 T1170 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
> [junit4:junit4] 2> 16228 T1170 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
> [junit4:junit4] 2> 16229 T1170 oass.IndexSchema.readSchema Reading Solr Schema
> [junit4:junit4] 2> 16240 T1170 oass.IndexSchema.readSchema Schema name=test
> [junit4:junit4] 2> 16645 T1170 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
> [junit4:junit4] 2> 16658 T1170 oass.IndexSchema.readSchema default search field in schema is text
> [junit4:junit4] 2> 16661 T1170 oass.IndexSchema.readSchema unique key field: id
> [junit4:junit4] 2> 16670 T1170 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
> [junit4:junit4] 2> 16673 T1170 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
> [junit4:junit4] 2> 16676 T1170 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
> [junit4:junit4] 2> 16677 T1170 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
> [junit4:junit4] 2> 16677 T1170 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
> [junit4:junit4] 2> 16677 T1170 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
> [junit4:junit4] 2> 16678 T1170 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
> [junit4:junit4] 2> 16679 T1170 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
> [junit4:junit4] 2> 16679 T1170 oasc.SolrCore.<init> [onenodecollectioncore] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1362353599259/onenodecollectioncore/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1362353595885/onenodecollection/
> [junit4:junit4] 2> 16679 T1170 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@64b87b
> [junit4:junit4] 2> 16680 T1170 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
> [junit4:junit4] 2> 16680 T1170 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1362353595885/onenodecollection forceNew:false
> [junit4:junit4] 2> 16681 T1170 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1362353595885/onenodecollection
> [junit4:junit4] 2> 16681 T1170 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1362353595885/onenodecollection/index/
> [junit4:junit4] 2> 16681 T1170 oasc.SolrCore.initIndex WARNING [onenodecollectioncore] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1362353595885/onenodecollection/index' doesn't exist. Creating new index...
> [junit4:junit4] 2> 16682 T1170 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1362353595885/onenodecollection/index forceNew:false
> [junit4:junit4] 2> 16683 T1170 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
> [junit4:junit4] 2> commit{dir=MockDirWrapper(NRTCachingDirectory(org.apache.lucene.store.RAMDirectory@17e95db lockFactory=org.apache.lucene.store.NativeFSLockFactory@1b4d3a1; maxCacheMB=0.4580078125 maxMergeSizeMB=0.19140625)),segFN=segments_1,generation=1,filenames=[segments_1]
> [junit4:junit4] 2> 16683 T1170 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
> [junit4:junit4] 2> 16684 T1170 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1362353595885/onenodecollection/index
> [junit4:junit4] 2> 16685 T1170 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
> [junit4:junit4] 2> 16685 T1170 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
> [junit4:junit4] 2> 16685 T1170 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
> [junit4:junit4] 2> 16686 T1170 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
> [junit4:junit4] 2> 16686 T1170 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
> [junit4:junit4] 2> 16687 T1170 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
> [junit4:junit4] 2> 16687 T1170 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
> [junit4:junit4] 2> 16687 T1170 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
> [junit4:junit4] 2> 16688 T1170 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
> [junit4:junit4] 2> 16690 T1170 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
> [junit4:junit4] 2> 16692 T1170 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1362353595885/onenodecollection
> [junit4:junit4] 2> 16693 T1170 oass.SolrIndexSearcher.<init> Opening Searcher@13a9497 main
> [junit4:junit4] 2> 16693 T1170 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1362353595885/onenodecollection/tlog
> [junit4:junit4] 2> 16694 T1170 oasu.CommitTracker.<init> Hard AutoCommit: disabled
> [junit4:junit4] 2> 16694 T1170 oasu.CommitTracker.<init> Soft AutoCommit: disabled
> [junit4:junit4] 2> 16697 T1170 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1362353595885/onenodecollection
> [junit4:junit4] 2> 16699 T1211 oasc.SolrCore.registerSearcher [onenodecollectioncore] Registered new searcher Searcher@13a9497 main{StandardDirectoryReader(segments_1:1)}
> [junit4:junit4] 2> 16700 T1170 oasc.ZkController.publish publishing core=onenodecollectioncore state=down
> [junit4:junit4] 2> 16927 T1155 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 16928 T1155 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
> [junit4:junit4] 2> "roles":null,
> [junit4:junit4] 2> "operation":"state",
> [junit4:junit4] 2> "collection":"collection1",
> [junit4:junit4] 2> "shard":"shard2",
> [junit4:junit4] 2> "core":"collection1",
> [junit4:junit4] 2> "base_url":"http://127.0.0.1:59775/ds_",
> [junit4:junit4] 2> "state":"recovering",
> [junit4:junit4] 2> "core_node_name":"4",
> [junit4:junit4] 2> "numShards":"2",
> [junit4:junit4] 2> "node_name":"127.0.0.1:59775_ds_"}
> [junit4:junit4] 2> 16931 T1155 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
> [junit4:junit4] 2> "roles":"none",
> [junit4:junit4] 2> "operation":"state",
> [junit4:junit4] 2> "collection":"onenodecollection",
> [junit4:junit4] 2> "shard":null,
> [junit4:junit4] 2> "core":"onenodecollectioncore",
> [junit4:junit4] 2> "base_url":"http://127.0.0.1:58158/ds_",
> [junit4:junit4] 2> "state":"down",
> [junit4:junit4] 2> "core_node_name":null,
> [junit4:junit4] 2> "numShards":"1",
> [junit4:junit4] 2> "node_name":"127.0.0.1:58158_ds_"}
> [junit4:junit4] 2> 16931 T1155 oasc.Overseer$ClusterStateUpdater.createCollection Create collection onenodecollection with numShards 1
> [junit4:junit4] 2> 16931 T1155 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
> [junit4:junit4] 2> 17156 T1174 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 17156 T1190 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 17156 T1161 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 17156 T1154 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 17156 T1206 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 17702 T1170 oasc.CoreContainer.registerCore registering core: onenodecollectioncore
> [junit4:junit4] 2> 17703 T1170 oasc.ZkController.register Register replica - core:onenodecollectioncore address:http://127.0.0.1:58158/ds_ collection:onenodecollection shard:shard1
> [junit4:junit4] 2> 17704 T1170 oascc.SolrZkClient.makePath makePath: /collections/onenodecollection/leader_elect/shard1/election
> [junit4:junit4] 2> 17709 T1137 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d329a25f00006 type:delete cxid:0x9f zxid:0xac txntype:-1 reqpath:n/a Error Path:/solr/collections/onenodecollection/leaders Error:KeeperErrorCode = NoNode for /solr/collections/onenodecollection/leaders
> [junit4:junit4] 2> 17710 T1170 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
> [junit4:junit4] 2> 17710 T1137 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d329a25f00006 type:create cxid:0xa0 zxid:0xad txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4] 2> 17711 T1170 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
> [junit4:junit4] 2> 17712 T1170 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
> [junit4:junit4] 2> 17712 T1170 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:58158/ds_/onenodecollectioncore/
> [junit4:junit4] 2> 17712 T1170 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
> [junit4:junit4] 2> 17713 T1170 oasc.SyncStrategy.syncToMe http://127.0.0.1:58158/ds_/onenodecollectioncore/ has no replicas
> [junit4:junit4] 2> 17713 T1170 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:58158/ds_/onenodecollectioncore/
> [junit4:junit4] 2> 17713 T1170 oascc.SolrZkClient.makePath makePath: /collections/onenodecollection/leaders/shard1
> [junit4:junit4] 2> 17717 T1137 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d329a25f00006 type:create cxid:0xaa zxid:0xb1 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4] 2> 18661 T1155 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 18666 T1161 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 18666 T1174 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 18666 T1206 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 18666 T1154 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 18666 T1190 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 18671 T1170 oasc.ZkController.register We are http://127.0.0.1:58158/ds_/onenodecollectioncore/ and leader is http://127.0.0.1:58158/ds_/onenodecollectioncore/
> [junit4:junit4] 2> 18672 T1170 oasc.ZkController.register No LogReplay needed for core=onenodecollectioncore baseURL=http://127.0.0.1:58158/ds_
> [junit4:junit4] 2> 18672 T1170 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
> [junit4:junit4] 2> 18673 T1170 oasc.ZkController.publish publishing core=onenodecollectioncore state=active
> [junit4:junit4] 2> 18674 T1170 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 18675 T1132 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: onenodecollection failOnTimeout:true timeout (sec):230
> [junit4:junit4] 2> 18676 T1132 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 19678 T1132 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> C55_STATE=coll:collection1 core:collection1 props:{collection=collection1, shard=shard2, core=collection1, base_url=http://127.0.0.1:59775/ds_, state=recovering, node_name=127.0.0.1:59775_ds_}
> [junit4:junit4] 2> 20121 T1209 C55 P59775 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:58158/ds_/collection1/ core=collection1 - recoveringAfterStartup=true
> [junit4:junit4] 2> 20122 T1209 C55 P59775 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:59775/ds_ START replicas=[http://127.0.0.1:58158/ds_/collection1/] nUpdates=100
> [junit4:junit4] 2> 20122 T1209 C55 P59775 oasu.PeerSync.sync WARNING no frame of reference to tell of we've missed updates
> [junit4:junit4] 2> 20123 T1209 C55 P59775 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
> [junit4:junit4] 2> 20123 T1209 C55 P59775 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
> [junit4:junit4] 2> 20123 T1209 C55 P59775 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
> [junit4:junit4] 2> 20123 T1209 C55 P59775 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
> [junit4:junit4] 2> 20123 T1209 C55 P59775 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:58158/ds_/collection1/. core=collection1
> [junit4:junit4] 2> 20124 T1209 C55 P59775 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
> [junit4:junit4] 2> ASYNC NEW_CORE C56 name=collection1 org.apache.solr.core.SolrCore@1f8b671 url=http://127.0.0.1:58158/ds_/collection1 node=127.0.0.1:58158_ds_ C56_STATE=coll:collection1 core:collection1 props:{collection=collection1, shard=shard2, core=collection1, base_url=http://127.0.0.1:58158/ds_, state=active, node_name=127.0.0.1:58158_ds_, leader=true}
> [junit4:junit4] 2> 20127 T1168 C56 P58158 REQ /get {version=2&distrib=false&getVersions=100&wt=javabin&qt=/get} status=0 QTime=0
> [junit4:junit4] 2> 20130 T1170 C56 P58158 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
> [junit4:junit4] 2> 20130 T1170 C56 P58158 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty1
> [junit4:junit4] 2> 20131 T1170 C56 P58158 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
> [junit4:junit4] 2> commit{dir=MockDirWrapper(NRTCachingDirectory(org.apache.lucene.store.RAMDirectory@46ba6b lockFactory=org.apache.lucene.store.NativeFSLockFactory@227076; maxCacheMB=0.4580078125 maxMergeSizeMB=0.19140625)),segFN=segments_1,generation=1,filenames=[segments_1]
> [junit4:junit4] 2> 20132 T1170 C56 P58158 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
> [junit4:junit4] 2> 20133 T1170 C56 P58158 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
> [junit4:junit4] 2> commit{dir=MockDirWrapper(NRTCachingDirectory(org.apache.lucene.store.RAMDirectory@46ba6b lockFactory=org.apache.lucene.store.NativeFSLockFactory@227076; maxCacheMB=0.4580078125 maxMergeSizeMB=0.19140625)),segFN=segments_1,generation=1,filenames=[segments_1]
> [junit4:junit4] 2> commit{dir=MockDirWrapper(NRTCachingDirectory(org.apache.lucene.store.RAMDirectory@46ba6b lockFactory=org.apache.lucene.store.NativeFSLockFactory@227076; maxCacheMB=0.4580078125 maxMergeSizeMB=0.19140625)),segFN=segments_2,generation=2,filenames=[segments_2]
> [junit4:junit4] 2> 20133 T1170 C56 P58158 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
> [junit4:junit4] 2> 20133 T1170 C56 P58158 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty1
> [junit4:junit4] 2> 20134 T1170 C56 P58158 oass.SolrIndexSearcher.<init> Opening Searcher@bdb4bc realtime
> [junit4:junit4] 2> 20134 T1170 C56 P58158 oasu.DirectUpdateHandler2.commit end_commit_flush
> [junit4:junit4] 2> 20134 T1170 C56 P58158 /update {version=2&softCommit=false&commit=true&wt=javabin&waitSearcher=true&openSearcher=false&commit_end_point=true} {commit=} 0 4
> [junit4:junit4] 2> 20135 T1209 C55 P59775 oash.ReplicationHandler.inform Commits will be reserved for 10000
> [junit4:junit4] 2> 20135 T1209 C55 P59775 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
> [junit4:junit4] 2> 20142 T1168 C56 P58158 oash.ReplicationHandler.inform Commits will be reserved for 10000
> [junit4:junit4] 2> 20143 T1168 C56 P58158 REQ /replication {version=2&command=indexversion&wt=javabin&qt=/replication} status=0 QTime=7
> [junit4:junit4] 2> 20143 T1209 C55 P59775 oash.SnapPuller.fetchLatestIndex Master's generation: 2
> [junit4:junit4] 2> 20144 T1209 C55 P59775 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
> [junit4:junit4] 2> 20144 T1209 C55 P59775 oash.SnapPuller.fetchLatestIndex Starting replication process
> [junit4:junit4] 2> 20146 T1167 C56 P58158 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty1
> [junit4:junit4] 2> 20146 T1167 C56 P58158 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty1/index
> [junit4:junit4] 2> 20146 T1167 C56 P58158 REQ /replication {version=2&command=filelist&generation=2&wt=javabin&qt=/replication} status=0 QTime=0
> [junit4:junit4] 2> 20147 T1209 C55 P59775 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
> [junit4:junit4] 2> 20148 T1209 C55 P59775 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty3/index.20130303153336029 forceNew:false
> [junit4:junit4] 2> 20148 T1209 C55 P59775 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty3
> [junit4:junit4] 2> 20149 T1209 C55 P59775 oash.SnapPuller.fetchLatestIndex Starting download to MockDirWrapper(org.apache.lucene.store.RAMDirectory@d8e0a7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7fe0e8) fullCopy=false
> [junit4:junit4] 2> 20151 T1170 C56 P58158 REQ /replication {file=segments_2&command=filecontent&generation=2&wt=filestream&checksum=true&qt=/replication} status=0 QTime=0
> [junit4:junit4] 2> 20152 T1209 C55 P59775 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty3/index
> [junit4:junit4] 2> 20152 T1209 C55 P59775 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
> [junit4:junit4] 2> 20153 T1209 C55 P59775 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty3
> [junit4:junit4] 2> 20153 T1209 C55 P59775 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty3
> [junit4:junit4] 2> 20154 T1209 C55 P59775 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
> [junit4:junit4] 2> 20154 T1209 C55 P59775 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
> [junit4:junit4] 2> 20154 T1209 C55 P59775 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty3
> [junit4:junit4] 2> 20155 T1209 C55 P59775 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=2
> [junit4:junit4] 2> commit{dir=MockDirWrapper(NRTCachingDirectory(org.apache.lucene.store.RAMDirectory@1b71845 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1bd6084; maxCacheMB=0.4580078125 maxMergeSizeMB=0.19140625)),segFN=segments_1,generation=1,filenames=[segments_1]
> [junit4:junit4] 2> commit{dir=MockDirWrapper(NRTCachingDirectory(org.apache.lucene.store.RAMDirectory@1b71845 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1bd6084; maxCacheMB=0.4580078125 maxMergeSizeMB=0.19140625)),segFN=segments_2,generation=2,filenames=[segments_2]
> [junit4:junit4] 2> 20156 T1209 C55 P59775 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
> [junit4:junit4] 2> 20156 T1209 C55 P59775 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
> [junit4:junit4] 2> 20156 T1209 C55 P59775 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty3
> [junit4:junit4] 2> 20157 T1209 C55 P59775 oass.SolrIndexSearcher.<init> Opening Searcher@3a9b12 main
> [junit4:junit4] 2> 20157 T1208 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3a9b12 main{StandardDirectoryReader(segments_2:1:nrt)}
> [junit4:junit4] 2> 20158 T1208 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty3/index
> [junit4:junit4] 2> 20158 T1209 C55 P59775 oash.SnapPuller.fetchLatestIndex removing temporary index download directory files MockDirWrapper(org.apache.lucene.store.RAMDirectory@d8e0a7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7fe0e8)
> [junit4:junit4] 2> 20158 T1209 C55 P59775 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty3/index.20130303153336029
> [junit4:junit4] 2> 20159 T1209 C55 P59775 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty3/index
> [junit4:junit4] 2> 20159 T1209 C55 P59775 oasc.RecoveryStrategy.replay No replay needed. core=collection1
> [junit4:junit4] 2> 20159 T1209 C55 P59775 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
> [junit4:junit4] 2> 20159 T1209 C55 P59775 oasc.ZkController.publish publishing core=collection1 state=active
> [junit4:junit4] 2> 20159 T1209 C55 P59775 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
> [junit4:junit4] 2> 20161 T1209 C55 P59775 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
> [junit4:junit4] 2> 20170 T1155 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 20171 T1155 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
> [junit4:junit4] 2> "roles":"none",
> [junit4:junit4] 2> "operation":"state",
> [junit4:junit4] 2> "collection":"onenodecollection",
> [junit4:junit4] 2> "shard":"shard1",
> [junit4:junit4] 2> "core":"onenodecollectioncore",
> [junit4:junit4] 2> "base_url":"http://127.0.0.1:58158/ds_",
> [junit4:junit4] 2> "state":"active",
> [junit4:junit4] 2> "core_node_name":null,
> [junit4:junit4] 2> "numShards":"1",
> [junit4:junit4] 2> "node_name":"127.0.0.1:58158_ds_"}
> [junit4:junit4] 2> 20174 T1155 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
> [junit4:junit4] 2> "roles":null,
> [junit4:junit4] 2> "operation":"state",
> [junit4:junit4] 2> "collection":"collection1",
> [junit4:junit4] 2> "shard":"shard2",
> [junit4:junit4] 2> "core":"collection1",
> [junit4:junit4] 2> "base_url":"http://127.0.0.1:59775/ds_",
> [junit4:junit4] 2> "state":"active",
> [junit4:junit4] 2> "core_node_name":"4",
> [junit4:junit4] 2> "numShards":"2",
> [junit4:junit4] 2> "node_name":"127.0.0.1:59775_ds_"}
> [junit4:junit4] 2> 20256 T1174 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 20256 T1206 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 20256 T1190 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 20256 T1161 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 20256 T1154 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 20679 T1132 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 20680 T1132 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: onenodecollection
> [junit4:junit4] 2> 20681 T1132 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
> [junit4:junit4] 2> 20708 T1132 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:53275 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@18d19bc
> [junit4:junit4] 2> 20709 T1132 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4:junit4] 2> 20709 T1214 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:53275. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
> [junit4:junit4] 2> 20709 T1214 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:53275, initiating session
> [junit4:junit4] 2> 20709 T1134 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:44008
> [junit4:junit4] 2> 20710 T1134 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:44008
> [junit4:junit4] 2> 20711 T1136 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d329a25f0000b with negotiated timeout 10000 for client /127.0.0.1:44008
> [junit4:junit4] 2> 20711 T1214 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:53275, sessionid = 0x13d329a25f0000b, negotiated timeout = 10000
> [junit4:junit4] 2> 20711 T1215 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@18d19bc name:ZooKeeperConnection Watcher:127.0.0.1:53275 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4:junit4] 2> 20711 T1132 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4:junit4] 2> 20740 T1137 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d329a25f0000b
> [junit4:junit4] 2> 20741 T1215 oaz.ClientCnxn$EventThread.run EventThread shut down
> [junit4:junit4] 2> 20741 T1134 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:44008 which had sessionid 0x13d329a25f0000b
> [junit4:junit4] 2> 20741 T1132 oaz.ZooKeeper.close Session: 0x13d329a25f0000b closed
> [junit4:junit4] 2> 20741 T1137 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d329a25f00004
> [junit4:junit4] 2> 20742 T1161 oaz.ClientCnxn$EventThread.run EventThread shut down
> [junit4:junit4] 2> 20742 T1132 oaz.ZooKeeper.close Session: 0x13d329a25f00004 closed
> [junit4:junit4] 2> 20742 T1134 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:43974 which had sessionid 0x13d329a25f00004
> [junit4:junit4] 2> 20744 T1132 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 35276
> [junit4:junit4] 2> 20744 T1132 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=20508092
> [junit4:junit4] 2> 21762 T1155 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 21762 T1155 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
> [junit4:junit4] 2> "roles":null,
> [junit4:junit4] 2> "operation":"state",
> [junit4:junit4] 2> "collection":"control_collection",
> [junit4:junit4] 2> "shard":"shard1",
> [junit4:junit4] 2> "core":"collection1",
> [junit4:junit4] 2> "base_url":"http://127.0.0.1:35276/ds_",
> [junit4:junit4] 2> "state":"down",
> [junit4:junit4] 2> "core_node_name":"1",
> [junit4:junit4] 2> "node_name":"127.0.0.1:35276_ds_"}
> [junit4:junit4] 2> 21765 T1154 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 21765 T1190 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 21765 T1206 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 21766 T1174 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 23747 T1132 oasc.ZkController.publish publishing core=collection1 state=down
> [junit4:junit4] 2> 23748 T1132 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
> [junit4:junit4] 2> 23768 T1132 oasc.SolrCore.close [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@d9acb9
> [junit4:junit4] 2> 23771 T1132 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty3/index
> [junit4:junit4] 2> 23772 T1132 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty3
> [junit4:junit4] 2> 23773 T1132 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
> [junit4:junit4] 2> 23773 T1132 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
> [junit4:junit4] 2> 23773 T1132 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
> [junit4:junit4] 2> 23774 T1132 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
> [junit4:junit4] 2> 23774 T1132 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
> [junit4:junit4] 2> 23775 T1132 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/control/data/index
> [junit4:junit4] 2> 23775 T1132 oasc.CachingDirectoryFactory.close Closing directory when closing factory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/control/data
> [junit4:junit4] 2> 23775 T1132 oasc.CachingDirectoryFactory.closeDirectory Closing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/control/data
> [junit4:junit4] 2> 23776 T1132 oasc.CachingDirectoryFactory.close Closing directory when closing factory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/control/data/index
> [junit4:junit4] 2> 23776 T1132 oasc.CachingDirectoryFactory.closeDirectory Closing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/control/data/index
> [junit4:junit4] 2> 23777 T1155 oasc.Overseer$ClusterStateUpdater.amILeader According to ZK I (id=89283205260115971-127.0.0.1:35276_ds_-n_0000000000) am no longer a leader.
> [junit4:junit4] 2> 23778 T1137 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d329a25f00003
> [junit4:junit4] 2> 23779 T1206 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 23779 T1134 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:43971 which had sessionid 0x13d329a25f00003
> [junit4:junit4] 2> 23779 T1190 oascc.ZkStateReader$3.process Updating live nodes... (3)
> [junit4:junit4] 2> 23779 T1174 oascc.ZkStateReader$3.process Updating live nodes... (3)
> [junit4:junit4] 2> 23780 T1190 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
> [junit4:junit4] 2> 23779 T1132 oaz.ZooKeeper.close Session: 0x13d329a25f00003 closed
> [junit4:junit4] 2> 23780 T1174 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
> [junit4:junit4] 2> 23780 T1206 oascc.ZkStateReader$3.process Updating live nodes... (3)
> [junit4:junit4] 2> 23781 T1137 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d329a25f00006 type:delete cxid:0xba zxid:0xcd txntype:-1 reqpath:n/a Error Path:/solr/overseer_elect/leader Error:KeeperErrorCode = NoNode for /solr/overseer_elect/leader
> [junit4:junit4] 2> 23782 T1174 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
> [junit4:junit4] 2> 23783 T1174 oasc.Overseer.start Overseer (id=89283205260115974-127.0.0.1:58158_ds_-n_0000000001) starting
> [junit4:junit4] 2> 23784 T1137 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d329a25f00006 type:create cxid:0xbf zxid:0xcf txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4] 2> 23784 T1137 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d329a25f00006 type:create cxid:0xc0 zxid:0xd0 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4] 2> 23785 T1137 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d329a25f00006 type:create cxid:0xc1 zxid:0xd1 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4] 2> 23786 T1137 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d329a25f00006 type:create cxid:0xc2 zxid:0xd2 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4] 2> 23786 T1217 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
> [junit4:junit4] 2> 23795 T1216 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
> [junit4:junit4] 2> 23796 T1216 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 23796 T1216 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
> [junit4:junit4] 2> "roles":null,
> [junit4:junit4] 2> "operation":"state",
> [junit4:junit4] 2> "collection":"control_collection",
> [junit4:junit4] 2> "shard":"shard1",
> [junit4:junit4] 2> "core":"collection1",
> [junit4:junit4] 2> "base_url":"http://127.0.0.1:35276/ds_",
> [junit4:junit4] 2> "state":"down",
> [junit4:junit4] 2> "core_node_name":"1",
> [junit4:junit4] 2> "numShards":"2",
> [junit4:junit4] 2> "node_name":"127.0.0.1:35276_ds_"}
> [junit4:junit4] 2> 23799 T1206 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4:junit4] 2> 23799 T1174 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4:junit4] 2> 23799 T1190 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4:junit4] 2> 23805 T1132 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/ds_,null}
> [junit4:junit4] 2> 23856 T1132 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 58158
> [junit4:junit4] 2> 23856 T1132 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=7335748
> [junit4:junit4] 2> 25079 T1154 oascc.ZkStateReader$3.process WARNING ZooKeeper watch triggered, but Solr cannot talk to ZK
> [junit4:junit4] 2> 25079 T1154 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 25079 T1154 oascc.ZkStateReader$2.process WARNING ZooKeeper watch triggered, but Solr cannot talk to ZK
> [junit4:junit4] 2> 25079 T1154 oaz.ClientCnxn$EventThread.run EventThread shut down
> [junit4:junit4] 2> 25302 T1216 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 25302 T1216 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
> [junit4:junit4] 2> "roles":"none",
> [junit4:junit4] 2> "operation":"state",
> [junit4:junit4] 2> "collection":"onenodecollection",
> [junit4:junit4] 2> "shard":"shard1",
> [junit4:junit4] 2> "core":"onenodecollectioncore",
> [junit4:junit4] 2> "base_url":"http://127.0.0.1:58158/ds_",
> [junit4:junit4] 2> "state":"down",
> [junit4:junit4] 2> "core_node_name":"127.0.0.1:58158_ds__onenodecollectioncore",
> [junit4:junit4] 2> "node_name":"127.0.0.1:58158_ds_"}
> [junit4:junit4] 2> 25305 T1216 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
> [junit4:junit4] 2> "roles":null,
> [junit4:junit4] 2> "operation":"state",
> [junit4:junit4] 2> "collection":"collection1",
> [junit4:junit4] 2> "shard":"shard2",
> [junit4:junit4] 2> "core":"collection1",
> [junit4:junit4] 2> "base_url":"http://127.0.0.1:58158/ds_",
> [junit4:junit4] 2> "state":"down",
> [junit4:junit4] 2> "core_node_name":"2",
> [junit4:junit4] 2> "node_name":"127.0.0.1:58158_ds_"}
> [junit4:junit4] 2> 25307 T1174 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4:junit4] 2> 25307 T1190 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4:junit4] 2> 25307 T1206 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4:junit4] 2> 26858 T1132 oasc.ZkController.publish publishing core=collection1 state=down
> [junit4:junit4] 2> 26859 T1132 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
> [junit4:junit4] 2> 26860 T1132 oasc.ZkController.publish publishing core=onenodecollectioncore state=down
> [junit4:junit4] 2> 26862 T1132 oasc.SolrCore.close [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@1f8b671
> [junit4:junit4] 2> 26865 T1132 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty3/index
> [junit4:junit4] 2> 26866 T1132 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty3
> [junit4:junit4] 2> 26866 T1132 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=1,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
> [junit4:junit4] 2> 26867 T1132 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
> [junit4:junit4] 2> 26867 T1132 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
> [junit4:junit4] 2> 26867 T1132 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
> [junit4:junit4] 2> C56_STATE=coll:collection1 core:collection1 props:{collection=collection1, shard=shard2, core=collection1, base_url=http://127.0.0.1:58158/ds_, state=down, node_name=127.0.0.1:58158_ds_, leader=true}
> [junit4:junit4] 2> 26868 T1132 C56 P58158 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty1/index
> [junit4:junit4] 2> 26869 T1132 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
> [junit4:junit4] 2> 26869 T1132 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty1/index
> [junit4:junit4] 2> 26869 T1132 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty1/index
> [junit4:junit4] 2> 26870 T1132 oasc.CachingDirectoryFactory.close Closing directory when closing factory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty1/index
> [junit4:junit4] 2> 26870 T1132 oasc.CachingDirectoryFactory.closeDirectory Closing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty1/index
> [junit4:junit4] 2> 26870 T1132 oasc.CachingDirectoryFactory.close Closing directory when closing factory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty1
> [junit4:junit4] 2> 26871 T1132 oasc.CachingDirectoryFactory.closeDirectory Closing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty1
> [junit4:junit4] 2> 26871 T1132 oasc.SolrCore.close [onenodecollectioncore] CLOSING SolrCore org.apache.solr.core.SolrCore@1ab038c
> [junit4:junit4] 2> 26875 T1132 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
> [junit4:junit4] 2> 26876 T1132 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
> [junit4:junit4] 2> 26876 T1132 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
> [junit4:junit4] 2> 26876 T1132 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
> [junit4:junit4] 2> 26877 T1132 oasc.SolrCore.closeSearcher [onenodecollectioncore] Closing main searcher on request.
> [junit4:junit4] 2> 26877 T1132 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1362353595885/onenodecollection/index
> [junit4:junit4] 2> 26878 T1132 oasc.CachingDirectoryFactory.close Closing directory when closing factory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1362353595885/onenodecollection
> [junit4:junit4] 2> 26878 T1132 oasc.CachingDirectoryFactory.closeDirectory Closing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1362353595885/onenodecollection
> [junit4:junit4] 2> 26878 T1132 oasc.CachingDirectoryFactory.close Closing directory when closing factory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1362353595885/onenodecollection/index
> [junit4:junit4] 2> 26879 T1132 oasc.CachingDirectoryFactory.closeDirectory Closing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1362353595885/onenodecollection/index
> [junit4:junit4] 2> 26879 T1216 oasc.Overseer$ClusterStateUpdater.amILeader According to ZK I (id=89283205260115974-127.0.0.1:58158_ds_-n_0000000001) am no longer a leader.
> [junit4:junit4] 2> 26880 T1137 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d329a25f00006
> [junit4:junit4] 2> 26881 T1134 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:43976 which had sessionid 0x13d329a25f00006
> [junit4:junit4] 2> 26881 T1132 oaz.ZooKeeper.close Session: 0x13d329a25f00006 closed
> [junit4:junit4] 2> 26881 T1137 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d329a25f0000a type:delete cxid:0x53 zxid:0xe3 txntype:-1 reqpath:n/a Error Path:/solr/collections/collection1/leaders/shard2 Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders/shard2
> [junit4:junit4] 2> 26882 T1206 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
> [junit4:junit4] 2> 26882 T1137 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d329a25f00008 type:delete cxid:0x6e zxid:0xe4 txntype:-1 reqpath:n/a Error Path:/solr/overseer_elect/leader Error:KeeperErrorCode = NoNode for /solr/overseer_elect/leader
> [junit4:junit4] 2> 26882 T1137 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d329a25f0000a type:create cxid:0x54 zxid:0xe5 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4] 2> 26883 T1190 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
> [junit4:junit4] 2> 26884 T1206 oasc.ShardLeaderElectionContext.shouldIBeLeader Checking if I should try and be the leader.
> [junit4:junit4] 2> 26884 T1206 oasc.ShardLeaderElectionContext.shouldIBeLeader My last published State was Active, it's okay to be the leader.
> [junit4:junit4] 2> 26884 T1206 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
> [junit4:junit4] 2> 26884 T1206 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:59775/ds_/collection1/
> [junit4:junit4] 2> 26885 T1206 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:59775/ds_ START replicas=[http://127.0.0.1:58158/ds_/collection1/] nUpdates=100
> [junit4:junit4] 2> 26885 T1206 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:59775/ds_ DONE. We have no versions. sync failed.
> [junit4:junit4] 2> 26886 T1206 oasc.SyncStrategy.syncReplicas Leader's attempt to sync with shard failed, moving to the next canidate
> [junit4:junit4] 2> 26886 T1206 oasc.ShardLeaderElectionContext.runLeaderProcess We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
> [junit4:junit4] 2> 26886 T1206 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:59775/ds_/collection1/
> [junit4:junit4] 2> 26886 T1206 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
> [junit4:junit4] 2> 26888 T1190 oasc.Overseer.start Overseer (id=89283205260115976-127.0.0.1:49875_ds_-n_0000000002) starting
> [junit4:junit4] 2> 26888 T1137 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d329a25f00008 type:create cxid:0x73 zxid:0xe8 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4] 2> 26889 T1137 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d329a25f00008 type:create cxid:0x74 zxid:0xea txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4] 2> 26890 T1137 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d329a25f0000a type:create cxid:0x5c zxid:0xeb txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4] 2> 26890 T1137 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d329a25f00008 type:create cxid:0x75 zxid:0xec txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4] 2> 26891 T1137 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d329a25f00008 type:create cxid:0x76 zxid:0xee txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4] 2> 26891 T1206 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
> [junit4:junit4] 2> 26892 T1220 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
> [junit4:junit4] 2> 26893 T1190 oascc.ZkStateReader$3.process Updating live nodes... (2)
> [junit4:junit4] 2> 26893 T1190 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
> [junit4:junit4] 2> 26894 T1206 oascc.ZkStateReader$3.process Updating live nodes... (2)
> [junit4:junit4] 2> 26894 T1219 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
> [junit4:junit4] 2> 26895 T1219 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 26896 T1219 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
> [junit4:junit4] 2> "roles":null,
> [junit4:junit4] 2> "operation":"state",
> [junit4:junit4] 2> "collection":"collection1",
> [junit4:junit4] 2> "shard":"shard2",
> [junit4:junit4] 2> "core":"collection1",
> [junit4:junit4] 2> "base_url":"http://127.0.0.1:58158/ds_",
> [junit4:junit4] 2> "state":"down",
> [junit4:junit4] 2> "core_node_name":"2",
> [junit4:junit4] 2> "numShards":"2",
> [junit4:junit4] 2> "node_name":"127.0.0.1:58158_ds_"}
> [junit4:junit4] 2> 26898 T1219 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
> [junit4:junit4] 2> "roles":"none",
> [junit4:junit4] 2> "operation":"state",
> [junit4:junit4] 2> "collection":"onenodecollection",
> [junit4:junit4] 2> "shard":"shard1",
> [junit4:junit4] 2> "core":"onenodecollectioncore",
> [junit4:junit4] 2> "base_url":"http://127.0.0.1:58158/ds_",
> [junit4:junit4] 2> "state":"down",
> [junit4:junit4] 2> "core_node_name":null,
> [junit4:junit4] 2> "numShards":"1",
> [junit4:junit4] 2> "node_name":"127.0.0.1:58158_ds_"}
> [junit4:junit4] 2> 26903 T1132 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/ds_,null}
> [junit4:junit4] 2> 26946 T1190 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
> [junit4:junit4] 2> 26946 T1206 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
> [junit4:junit4] 2> 26955 T1132 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 49875
> [junit4:junit4] 2> 26955 T1132 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=9096842
> [junit4:junit4] 2> 27956 T1132 oasc.ZkController.publish publishing core=collection1 state=down
> [junit4:junit4] 2> 27957 T1132 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
> [junit4:junit4] 2> 27959 T1132 oasc.SolrCore.close [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@1c3b982
> [junit4:junit4] 2> 27962 T1132 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty3/index
> [junit4:junit4] 2> 27963 T1132 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty3
> [junit4:junit4] 2> 27963 T1132 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
> [junit4:junit4] 2> 27964 T1132 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
> [junit4:junit4] 2> 27964 T1132 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
> [junit4:junit4] 2> 27964 T1132 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
> [junit4:junit4] 2> 27965 T1132 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
> [junit4:junit4] 2> 27965 T1132 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty2/index
> [junit4:junit4] 2> 27966 T1132 oasc.CachingDirectoryFactory.close Closing directory when closing factory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty2/index
> [junit4:junit4] 2> 27966 T1132 oasc.CachingDirectoryFactory.closeDirectory Closing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty2/index
> [junit4:junit4] 2> 27966 T1132 oasc.CachingDirectoryFactory.close Closing directory when closing factory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty2
> [junit4:junit4] 2> 27967 T1132 oasc.CachingDirectoryFactory.closeDirectory Closing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty2
> [junit4:junit4] 2> 27967 T1219 oasc.Overseer$ClusterStateUpdater.amILeader According to ZK I (id=89283205260115976-127.0.0.1:49875_ds_-n_0000000002) am no longer a leader.
> [junit4:junit4] 2> 27968 T1137 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d329a25f00008
> [junit4:junit4] 2> 27969 T1206 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
> [junit4:junit4] 2> 27969 T1134 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:43980 which had sessionid 0x13d329a25f00008
> [junit4:junit4] 2> 27969 T1132 oaz.ZooKeeper.close Session: 0x13d329a25f00008 closed
> [junit4:junit4] 2> 27970 T1206 oascc.ZkStateReader$3.process Updating live nodes... (1)
> [junit4:junit4] 2> 27971 T1137 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d329a25f0000a type:delete cxid:0x67 zxid:0xff txntype:-1 reqpath:n/a Error Path:/solr/overseer_elect/leader Error:KeeperErrorCode = NoNode for /solr/overseer_elect/leader
> [junit4:junit4] 2> 27971 T1206 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
> [junit4:junit4] 2> 27973 T1206 oasc.Overseer.start Overseer (id=89283205260115978-127.0.0.1:59775_ds_-n_0000000003) starting
> [junit4:junit4] 2> 27973 T1137 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d329a25f0000a type:create cxid:0x6c zxid:0x101 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4] 2> 27974 T1137 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d329a25f0000a type:create cxid:0x6d zxid:0x102 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4] 2> 27974 T1137 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d329a25f0000a type:create cxid:0x6e zxid:0x103 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4] 2> 27975 T1137 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d329a25f0000a type:create cxid:0x6f zxid:0x104 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4] 2> 27975 T1222 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
> [junit4:junit4] 2> 27976 T1221 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
> [junit4:junit4] 2> 27978 T1221 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 27978 T1221 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
> [junit4:junit4] 2> "roles":null,
> [junit4:junit4] 2> "operation":"state",
> [junit4:junit4] 2> "collection":"collection1",
> [junit4:junit4] 2> "shard":"shard1",
> [junit4:junit4] 2> "core":"collection1",
> [junit4:junit4] 2> "base_url":"http://127.0.0.1:49875/ds_",
> [junit4:junit4] 2> "state":"down",
> [junit4:junit4] 2> "core_node_name":"3",
> [junit4:junit4] 2> "node_name":"127.0.0.1:49875_ds_"}
> [junit4:junit4] 2> 27981 T1221 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
> [junit4:junit4] 2> "roles":null,
> [junit4:junit4] 2> "operation":"state",
> [junit4:junit4] 2> "collection":"collection1",
> [junit4:junit4] 2> "shard":"shard1",
> [junit4:junit4] 2> "core":"collection1",
> [junit4:junit4] 2> "base_url":"http://127.0.0.1:49875/ds_",
> [junit4:junit4] 2> "state":"down",
> [junit4:junit4] 2> "core_node_name":"3",
> [junit4:junit4] 2> "numShards":"2",
> [junit4:junit4] 2> "node_name":"127.0.0.1:49875_ds_"}
> [junit4:junit4] 2> 27991 T1132 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/ds_,null}
> [junit4:junit4] 2> 28029 T1206 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
> [junit4:junit4] 2> 28044 T1132 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 59775
> [junit4:junit4] 2> 28044 T1132 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=3438750
> [junit4:junit4] 2> 28181 T1174 oascc.ZkStateReader$3.process WARNING ZooKeeper watch triggered, but Solr cannot talk to ZK
> [junit4:junit4] 2> 28181 T1174 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
> [junit4:junit4] 2> 28181 T1174 oascc.ZkStateReader$2.process WARNING ZooKeeper watch triggered, but Solr cannot talk to ZK
> [junit4:junit4] 2> 28181 T1174 oaz.ClientCnxn$EventThread.run EventThread shut down
> [junit4:junit4] 2> 29045 T1132 oasc.ZkController.publish publishing core=collection1 state=down
> [junit4:junit4] 2> 29046 T1132 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
> [junit4:junit4] 2> 29049 T1132 oasc.SolrCore.close [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@b9d6ee
> [junit4:junit4] 2> 29053 T1132 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty3/index
> [junit4:junit4] 2> 29054 T1132 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty3
> [junit4:junit4] 2> 29055 T1132 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty3/index
> [junit4:junit4] 2> 29055 T1132 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty3
> [junit4:junit4] 2> 29056 T1132 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
> [junit4:junit4] 2> 29056 T1132 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
> [junit4:junit4] 2> 29057 T1132 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
> [junit4:junit4] 2> 29057 T1132 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
> [junit4:junit4] 2> C55_STATE=coll:collection1 core:collection1 props:{collection=collection1, shard=shard2, core=collection1, base_url=http://127.0.0.1:59775/ds_, state=active, node_name=127.0.0.1:59775_ds_, leader=true}
> [junit4:junit4] 2> 29058 T1132 C55 P59775 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty3/index
> [junit4:junit4] 2> 29059 T1132 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
> [junit4:junit4] 2> 29059 T1132 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty3/index
> [junit4:junit4] 2> 29060 T1132 oasc.CachingDirectoryFactory.close Closing directory when closing factory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty3/index
> [junit4:junit4] 2> 29061 T1132 oasc.CachingDirectoryFactory.closeDirectory Closing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty3/index
> [junit4:junit4] 2> 29061 T1132 oasc.CachingDirectoryFactory.close Closing directory when closing factory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty3/index.20130303153336029
> [junit4:junit4] 2> 29062 T1132 oasc.CachingDirectoryFactory.closeDirectory Closing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty3/index.20130303153336029
> [junit4:junit4] 2> 29063 T1132 oasc.CachingDirectoryFactory.close Closing directory when closing factory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty3
> [junit4:junit4] 2> 29063 T1132 oasc.CachingDirectoryFactory.closeDirectory Closing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty3
> [junit4:junit4] 2> 29066 T1221 oasc.Overseer$ClusterStateUpdater.amILeader According to ZK I (id=89283205260115978-127.0.0.1:59775_ds_-n_0000000003) am no longer a leader.
> [junit4:junit4] 2> 29066 T1137 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d329a25f0000a
> [junit4:junit4] 2> 29087 T1206 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
> [junit4:junit4] 2> 29087 T1134 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:43989 which had sessionid 0x13d329a25f0000a
> [junit4:junit4] 2> 29087 T1132 oaz.ZooKeeper.close Session: 0x13d329a25f0000a closed
> [junit4:junit4] 2> 29087 T1206 oascc.ZkStateReader$2.process WARNING ZooKeeper watch triggered, but Solr cannot talk to ZK
> [junit4:junit4] 2> 29088 T1206 oascc.ZkStateReader$3.process WARNING ZooKeeper watch triggered, but Solr cannot talk to ZK
> [junit4:junit4] 2> 29088 T1206 oaz.ClientCnxn$EventThread.run EventThread shut down
> [junit4:junit4] 2> 29110 T1132 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/ds_,null}
> [junit4:junit4] 2> 29167 T1132 oas.SolrTestCaseJ4.tearDown ###Ending testDistribSearch
> [junit4:junit4] 2> 29168 T1132 oazs.ZooKeeperServer.shutdown shutting down
> [junit4:junit4] 2> 29168 T1132 oazs.SessionTrackerImpl.shutdown Shutting down
> [junit4:junit4] 2> 29168 T1132 oazs.PrepRequestProcessor.shutdown Shutting down
> [junit4:junit4] 2> 29169 T1137 oazs.PrepRequestProcessor.run PrepRequestProcessor exited loop!
> [junit4:junit4] 2> 29169 T1132 oazs.SyncRequestProcessor.shutdown Shutting down
> [junit4:junit4] 2> 29169 T1136 oazs.SyncRequestProcessor.run SyncRequestProcessor exited!
> [junit4:junit4] 2> 29170 T1132 oazs.FinalRequestProcessor.shutdown shutdown of request processor complete
> [junit4:junit4] 2> 29171 T1132 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:53275 53275
> [junit4:junit4] 2> 29173 T1134 oazs.NIOServerCnxnFactory.run NIOServerCnxn factory exited run method
> [junit4:junit4] 2> 29173 T1132 oazs.ZooKeeperServer.shutdown shutting down
> [junit4:junit4] 2> 29174 T1132 oazs.SessionTrackerImpl.shutdown Shutting down
> [junit4:junit4] 2> 29174 T1132 oazs.PrepRequestProcessor.shutdown Shutting down
> [junit4:junit4] 2> 29174 T1132 oazs.SyncRequestProcessor.shutdown Shutting down
> [junit4:junit4] 2> 29175 T1132 oazs.FinalRequestProcessor.shutdown shutdown of request processor complet
>
> [...truncated too long message...]
>
> Test(UnloadDistributedZkTest.java:77)
> [junit4:junit4] > at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:806)
> [junit4:junit4] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> [junit4:junit4] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> [junit4:junit4] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> [junit4:junit4] > at java.lang.reflect.Method.invoke(Method.java:487)
> [junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
> [junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
> [junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
> [junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
> [junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
> [junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
> [junit4:junit4] > at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
> [junit4:junit4] > at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
> [junit4:junit4] > at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
> [junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
> [junit4:junit4] > at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
> [junit4:junit4] > at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
> [junit4:junit4] > at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
> [junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> [junit4:junit4] > at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
> [junit4:junit4] > at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
> [junit4:junit4] > at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
> [junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
> [junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
> [junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
> [junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
> [junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> [junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
> [junit4:junit4] > at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
> [junit4:junit4] > at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
> [junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
> [junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
> [junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
> [junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> [junit4:junit4] > at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
> [junit4:junit4] > at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
> [junit4:junit4] > at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
> [junit4:junit4] > at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
> [junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> [junit4:junit4] > at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
> [junit4:junit4] > at java.lang.Thread.run(Thread.java:722)
> [junit4:junit4] 2> 136346 T2567 oas.SolrTestCaseJ4.deleteCore ###deleteCore
> [junit4:junit4] 2> 136357 T2566 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 3 leaked thread(s).
> [junit4:junit4] 2> 136639 T2570 oazs.SessionTrackerImpl.run SessionTrackerImpl exited loop!
> [junit4:junit4] 2> 137438 T2649 oascc.ZkStateReader$3.process WARNING ZooKeeper watch triggered, but Solr cannot talk to ZK
> [junit4:junit4] 2> 137438 T2649 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
> [junit4:junit4] 2> 137439 T2649 oascc.ZkStateReader$2.process WARNING ZooKeeper watch triggered, but Solr cannot talk to ZK
> [junit4:junit4] 2> 137439 T2649 oaz.ClientCnxn$EventThread.run EventThread shut down
> [junit4:junit4] 2> NOTE: test params are: codec=Lucene41, sim=DefaultSimilarity, locale=da_DK, timezone=SystemV/AST4ADT
> [junit4:junit4] 2> NOTE: Linux 3.2.0-38-generic i386/Oracle Corporation 1.8.0-ea (32-bit)/cpus=8,threads=2,free=36852112,total=145764352
> [junit4:junit4] 2> NOTE: All tests run in this JVM: [PrimitiveFieldTypeTest, TestStressVersions, SolrCmdDistributorTest, TestArbitraryIndexDir, SuggesterFSTTest, TestSolrIndexConfig, SuggesterWFSTTest, TestCodecSupport, TestFoldingMultitermQuery, SearchHandlerTest, TestSolrDiscoveryProperties, TestFiltering, TestSurroundQueryParser, DebugComponentTest, TestRandomDVFaceting, StatelessScriptUpdateProcessorFactoryTest, SpatialFilterTest, SolrRequestParserTest, TestZkChroot, TestPHPSerializedResponseWriter, LeaderElectionIntegrationTest, TestPseudoReturnFields, TestWordDelimiterFilterFactory, SampleTest, TestLazyCores, TestValueSourceCache, RAMDirectoryFactoryTest, TestSearchPerf, SolrPluginUtilsTest, SpellCheckComponentTest, MoreLikeThisHandlerTest, SuggesterTest, BadComponentTest, TestFuzzyAnalyzedSuggestions, SimpleFacetsTest, OverseerCollectionProcessorTest, TestIndexSearcher, SpellCheckCollatorTest, FileBasedSpellCheckerTest, UpdateRequestProcessorFactoryTest, TestSolrXMLSerializer, ShardRoutingCustomTest, SynonymTokenizerTest, DistributedSpellCheckComponentTest, SOLR749Test, TestPostingsSolrHighlighter, TestFaceting, ClusterStateTest, DateMathParserTest, TestBinaryResponseWriter, TestExtendedDismaxParser, TestSolr4Spatial, SyncSliceTest, TestOmitPositions, TestUpdate, BasicDistributedZk2Test, TestLMDirichletSimilarityFactory, TestAtomicUpdateErrorCases, TestMergePolicyConfig, DistanceFunctionTest, IndexBasedSpellCheckerTest, TestBinaryField, CoreAdminHandlerTest, RecoveryZkTest, FullSolrCloudDistribCmdsTest, TestReplicationHandler, TestRealTimeGet, ZkControllerTest, TestRandomFaceting, WordBreakSolrSpellCheckerTest, TestHashPartitioner, TestJoin, SortByFunctionTest, SignatureUpdateProcessorFactoryTest, DirectSolrConnectionTest, CacheHeaderTest, PolyFieldTest, MBeansHandlerTest, IndexSchemaTest, BinaryUpdateRequestHandlerTest, MinimalSchemaTest, XmlUpdateRequestHandlerTest, PingRequestHandlerTest, TermVectorComponentTest, DirectUpdateHandlerOptimizeTest, TestReversedWildcardFilterFactory, JSONWriterTest, TestQueryUtils, UniqFieldsUpdateProcessorFactoryTest, ShowFileRequestHandlerTest, TestPropInject, NoCacheHeaderTest, DefaultValueUpdateProcessorTest, TestQuerySenderListener, NumericFieldsTest, OutputWriterTest, TestConfig, TestPhraseSuggestions, LegacyHTMLStripCharFilterTest, TestDefaultSimilarityFactory, PluginInfoTest, ResourceLoaderTest, DateFieldTest, TestBadConfig, TestUtils, PreAnalyzedFieldTest, ZkNodePropsTest, CircularListTest, AnalysisAfterCoreReloadTest, TestDocumentBuilder, PathHierarchyTokenizerFactoryTest, TestCharFilters, ClusterStateUpdateTest, UnloadDistributedZkTest]
> [junit4:junit4] Completed on J0 in 137.64s, 1 test, 1 failure <<< FAILURES!
>
> [...truncated 75 lines...]
> BUILD FAILED
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:381: The following error occurred while executing this line:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:361: The following error occurred while executing this line:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:39: The following error occurred while executing this line:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build.xml:183: The following error occurred while executing this line:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/common-build.xml:449: The following error occurred while executing this line:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:1207: The following error occurred while executing this line:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:870: There were test failures: 261 suites, 1101 tests, 1 error, 1 failure, 13 ignored (7 assumptions)
>
> Total time: 41 minutes 12 seconds
> Build step 'Invoke Ant' marked build as failure
> Archiving artifacts
> Recording test results
> Description set: Java: 32bit/jdk1.8.0-ea-b79 -client -XX:+UseSerialGC
> Email was triggered for: Failure
> Sending email for trigger: Failure
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[JENKINS] Lucene-Solr-trunk-Linux (32bit/jdk1.8.0-ea-b79) - Build #
4558 - Still Failing!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/4558/
Java: 32bit/jdk1.8.0-ea-b79 -client -XX:+UseSerialGC
2 tests failed.
REGRESSION: org.apache.solr.cloud.BasicDistributedZk2Test.testDistribSearch
Error Message:
Server at http://127.0.0.1:49875/ds_/onenodecollectioncore returned non ok status:404, message:Can not find: /ds_/onenodecollectioncore/update
Stack Trace:
org.apache.solr.common.SolrException: Server at http://127.0.0.1:49875/ds_/onenodecollectioncore returned non ok status:404, message:Can not find: /ds_/onenodecollectioncore/update
at __randomizedtesting.SeedInfo.seed([E33CD887C055A93A:62DA569FB70AC906]:0)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:375)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:181)
at org.apache.solr.client.solrj.request.AbstractUpdateRequest.process(AbstractUpdateRequest.java:117)
at org.apache.solr.client.solrj.SolrServer.add(SolrServer.java:116)
at org.apache.solr.client.solrj.SolrServer.add(SolrServer.java:102)
at org.apache.solr.cloud.BasicDistributedZk2Test.testNodeWithoutCollectionForwarding(BasicDistributedZk2Test.java:200)
at org.apache.solr.cloud.BasicDistributedZk2Test.doTest(BasicDistributedZk2Test.java:89)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:806)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:487)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:722)
REGRESSION: org.apache.solr.cloud.UnloadDistributedZkTest.testDistribSearch
Error Message:
Still found shard
Stack Trace:
java.lang.AssertionError: Still found shard
at __randomizedtesting.SeedInfo.seed([E33CD887C055A93A:62DA569FB70AC906]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.UnloadDistributedZkTest.testUnloadShardAndCollection(UnloadDistributedZkTest.java:124)
at org.apache.solr.cloud.UnloadDistributedZkTest.doTest(UnloadDistributedZkTest.java:77)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:806)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:487)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:722)
Build Log:
[...truncated 9183 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.BasicDistributedZk2Test
[junit4:junit4] 2> 0 T1132 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /ds_/
[junit4:junit4] 2> 3 T1132 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1362353595885
[junit4:junit4] 2> 3 T1132 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4] 2> 4 T1133 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4] 2> 4 T1133 oazs.ZooKeeperServer.setTickTime tickTime set to 1000
[junit4:junit4] 2> 5 T1133 oazs.ZooKeeperServer.setMinSessionTimeout minSessionTimeout set to -1
[junit4:junit4] 2> 5 T1133 oazs.ZooKeeperServer.setMaxSessionTimeout maxSessionTimeout set to -1
[junit4:junit4] 2> 5 T1133 oazs.NIOServerCnxnFactory.configure binding to port 0.0.0.0/0.0.0.0:0
[junit4:junit4] 2> 6 T1133 oazsp.FileTxnSnapLog.save Snapshotting: 0x0 to /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/zookeeper/server1/data/version-2/snapshot.0
[junit4:junit4] 2> 104 T1132 oasc.ZkTestServer.run start zk server on port:53275
[junit4:junit4] 2> 104 T1132 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:53275 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@1b9eccd
[junit4:junit4] 2> 105 T1132 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 105 T1138 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:53275. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 106 T1138 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:53275, initiating session
[junit4:junit4] 2> 106 T1134 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:43968
[junit4:junit4] 2> 106 T1134 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:43968
[junit4:junit4] 2> 106 T1136 oazsp.FileTxnLog.append Creating new log file: log.1
[junit4:junit4] 2> 126 T1136 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d329a25f00000 with negotiated timeout 10000 for client /127.0.0.1:43968
[junit4:junit4] 2> 126 T1138 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:53275, sessionid = 0x13d329a25f00000, negotiated timeout = 10000
[junit4:junit4] 2> 127 T1139 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1b9eccd name:ZooKeeperConnection Watcher:127.0.0.1:53275 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 127 T1132 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 127 T1132 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4] 2> 130 T1137 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d329a25f00000
[junit4:junit4] 2> 130 T1139 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 131 T1134 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:43968 which had sessionid 0x13d329a25f00000
[junit4:junit4] 2> 130 T1132 oaz.ZooKeeper.close Session: 0x13d329a25f00000 closed
[junit4:junit4] 2> 131 T1132 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:53275/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@17e1b0a
[junit4:junit4] 2> 132 T1132 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 132 T1140 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:53275. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 133 T1140 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:53275, initiating session
[junit4:junit4] 2> 133 T1134 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:43969
[junit4:junit4] 2> 133 T1134 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:43969
[junit4:junit4] 2> 133 T1136 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d329a25f00001 with negotiated timeout 10000 for client /127.0.0.1:43969
[junit4:junit4] 2> 133 T1140 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:53275, sessionid = 0x13d329a25f00001, negotiated timeout = 10000
[junit4:junit4] 2> 134 T1141 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@17e1b0a name:ZooKeeperConnection Watcher:127.0.0.1:53275/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 134 T1132 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 134 T1132 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4] 2> 136 T1132 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4] 2> 137 T1132 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4] 2> 138 T1132 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4] 2> 140 T1132 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 140 T1132 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 143 T1132 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4:junit4] 2> 143 T1132 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4] 2> 146 T1132 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4] 2> 146 T1132 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4] 2> 147 T1132 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4] 2> 148 T1132 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4] 2> 149 T1132 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4] 2> 150 T1132 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4] 2> 152 T1132 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 152 T1132 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 153 T1132 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 154 T1132 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 158 T1132 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 159 T1132 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 161 T1132 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4] 2> 161 T1132 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4] 2> 163 T1137 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d329a25f00001
[junit4:junit4] 2> 163 T1141 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 163 T1134 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:43969 which had sessionid 0x13d329a25f00001
[junit4:junit4] 2> 163 T1132 oaz.ZooKeeper.close Session: 0x13d329a25f00001 closed
[junit4:junit4] 2> 222 T1132 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4] 2> 249 T1132 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:35276
[junit4:junit4] 2> 250 T1132 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 250 T1132 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 251 T1132 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1362353596046
[junit4:junit4] 2> 251 T1132 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1362353596046/solr.xml
[junit4:junit4] 2> 251 T1132 oasc.CoreContainer.<init> New CoreContainer 20508092
[junit4:junit4] 2> 252 T1132 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1362353596046/'
[junit4:junit4] 2> 252 T1132 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1362353596046/'
[junit4:junit4] 2> 269 T1132 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 270 T1132 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 270 T1132 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 270 T1132 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 271 T1132 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 271 T1132 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 271 T1132 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 272 T1132 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 272 T1132 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 272 T1132 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 279 T1132 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4] 2> 296 T1132 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:53275/solr
[junit4:junit4] 2> 297 T1132 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 297 T1132 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:53275 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@10883c6
[junit4:junit4] 2> 298 T1151 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:53275. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 298 T1132 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 299 T1151 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:53275, initiating session
[junit4:junit4] 2> 299 T1134 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:43970
[junit4:junit4] 2> 299 T1134 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:43970
[junit4:junit4] 2> 300 T1136 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d329a25f00002 with negotiated timeout 20000 for client /127.0.0.1:43970
[junit4:junit4] 2> 300 T1151 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:53275, sessionid = 0x13d329a25f00002, negotiated timeout = 20000
[junit4:junit4] 2> 300 T1152 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@10883c6 name:ZooKeeperConnection Watcher:127.0.0.1:53275 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 300 T1132 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 301 T1137 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d329a25f00002
[junit4:junit4] 2> 302 T1152 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 302 T1134 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:43970 which had sessionid 0x13d329a25f00002
[junit4:junit4] 2> 302 T1132 oaz.ZooKeeper.close Session: 0x13d329a25f00002 closed
[junit4:junit4] 2> 303 T1132 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 305 T1132 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:53275/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@12a91bf
[junit4:junit4] 2> 306 T1153 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:53275. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 306 T1132 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 306 T1153 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:53275, initiating session
[junit4:junit4] 2> 306 T1134 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:43971
[junit4:junit4] 2> 307 T1134 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:43971
[junit4:junit4] 2> 307 T1136 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d329a25f00003 with negotiated timeout 20000 for client /127.0.0.1:43971
[junit4:junit4] 2> 307 T1153 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:53275, sessionid = 0x13d329a25f00003, negotiated timeout = 20000
[junit4:junit4] 2> 308 T1154 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@12a91bf name:ZooKeeperConnection Watcher:127.0.0.1:53275/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 308 T1132 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 309 T1137 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d329a25f00003 type:create cxid:0x2 zxid:0x1a txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 310 T1132 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4] 2> 311 T1132 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:35276_ds_
[junit4:junit4] 2> 312 T1137 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d329a25f00003 type:delete cxid:0x8 zxid:0x1c txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:35276_ds_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:35276_ds_
[junit4:junit4] 2> 312 T1132 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:35276_ds_
[junit4:junit4] 2> 314 T1132 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4] 2> 316 T1137 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d329a25f00003 type:delete cxid:0x16 zxid:0x21 txntype:-1 reqpath:n/a Error Path:/solr/overseer_elect/leader Error:KeeperErrorCode = NoNode for /solr/overseer_elect/leader
[junit4:junit4] 2> 317 T1132 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4] 2> 319 T1132 oasc.Overseer.start Overseer (id=89283205260115971-127.0.0.1:35276_ds_-n_0000000000) starting
[junit4:junit4] 2> 319 T1137 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d329a25f00003 type:create cxid:0x1b zxid:0x23 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 320 T1137 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d329a25f00003 type:create cxid:0x1c zxid:0x24 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 320 T1137 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d329a25f00003 type:create cxid:0x1d zxid:0x25 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 321 T1137 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d329a25f00003 type:create cxid:0x1e zxid:0x26 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 322 T1156 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4] 2> 322 T1132 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4] 2> 324 T1132 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 325 T1155 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4] 2> 327 T1157 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1362353596046/collection1
[junit4:junit4] 2> 328 T1157 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4] 2> 328 T1157 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 328 T1157 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4] 2> 329 T1157 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1362353596046/collection1/'
[junit4:junit4] 2> 329 T1157 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1362353596046/collection1/lib/README' to classloader
[junit4:junit4] 2> 330 T1157 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1362353596046/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 355 T1157 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 395 T1157 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 396 T1157 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4] 2> 400 T1157 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4] 2> 802 T1157 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 807 T1157 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 810 T1157 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 819 T1157 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 822 T1157 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 825 T1157 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 826 T1157 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 826 T1157 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 827 T1157 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 828 T1157 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 828 T1157 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 828 T1157 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1362353596046/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/control/data/
[junit4:junit4] 2> 828 T1157 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@64b87b
[junit4:junit4] 2> 828 T1157 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 829 T1157 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/control/data forceNew:false
[junit4:junit4] 2> 829 T1157 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/control/data
[junit4:junit4] 2> 829 T1157 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/control/data/index/
[junit4:junit4] 2> 830 T1157 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/control/data/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 830 T1157 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/control/data/index forceNew:false
[junit4:junit4] 2> 831 T1157 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(NRTCachingDirectory(org.apache.lucene.store.RAMDirectory@53381c lockFactory=org.apache.lucene.store.NativeFSLockFactory@d4aa65; maxCacheMB=0.4580078125 maxMergeSizeMB=0.19140625)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> 831 T1157 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4] 2> 832 T1157 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/control/data/index
[junit4:junit4] 2> 832 T1157 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 833 T1157 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 833 T1157 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 833 T1157 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 834 T1157 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 834 T1157 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 834 T1157 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 834 T1157 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 835 T1157 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 837 T1157 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 839 T1157 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/control/data
[junit4:junit4] 2> 839 T1157 oass.SolrIndexSearcher.<init> Opening Searcher@47b90d main
[junit4:junit4] 2> 840 T1157 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/control/data/tlog
[junit4:junit4] 2> 840 T1157 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 840 T1157 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 842 T1157 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/control/data
[junit4:junit4] 2> 843 T1158 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@47b90d main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 843 T1157 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 844 T1157 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 844 T1137 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d329a25f00003 type:create cxid:0x50 zxid:0x29 txntype:-1 reqpath:n/a Error Path:/solr/overseer/queue Error:KeeperErrorCode = NoNode for /solr/overseer/queue
[junit4:junit4] 2> 1829 T1155 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1830 T1155 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:35276/ds_",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "core_node_name":"1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "node_name":"127.0.0.1:35276_ds_"}
[junit4:junit4] 2> 1830 T1155 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
[junit4:junit4] 2> 1830 T1155 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4] 2> 1831 T1137 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d329a25f00003 type:create cxid:0x58 zxid:0x2c txntype:-1 reqpath:n/a Error Path:/solr/overseer/queue-work Error:KeeperErrorCode = NoNode for /solr/overseer/queue-work
[junit4:junit4] 2> 1834 T1154 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 1846 T1157 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 1846 T1157 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:35276/ds_ collection:control_collection shard:shard1
[junit4:junit4] 2> 1847 T1157 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4:junit4] 2> 1853 T1137 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d329a25f00003 type:delete cxid:0x72 zxid:0x36 txntype:-1 reqpath:n/a Error Path:/solr/collections/control_collection/leaders Error:KeeperErrorCode = NoNode for /solr/collections/control_collection/leaders
[junit4:junit4] 2> 1853 T1157 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 1854 T1137 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d329a25f00003 type:create cxid:0x73 zxid:0x37 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 1855 T1157 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 1855 T1157 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 1856 T1157 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:35276/ds_/collection1/
[junit4:junit4] 2> 1856 T1157 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 1856 T1157 oasc.SyncStrategy.syncToMe http://127.0.0.1:35276/ds_/collection1/ has no replicas
[junit4:junit4] 2> 1856 T1157 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:35276/ds_/collection1/
[junit4:junit4] 2> 1856 T1157 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4:junit4] 2> 1859 T1137 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d329a25f00003 type:create cxid:0x7d zxid:0x3b txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 3338 T1155 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 3344 T1154 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 3364 T1157 oasc.ZkController.register We are http://127.0.0.1:35276/ds_/collection1/ and leader is http://127.0.0.1:35276/ds_/collection1/
[junit4:junit4] 2> 3364 T1157 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:35276/ds_
[junit4:junit4] 2> 3364 T1157 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 3365 T1157 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 3365 T1157 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 3366 T1157 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 3367 T1132 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
[junit4:junit4] 2> 3367 T1132 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 3368 T1132 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 3371 T1132 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4] 2> 3372 T1132 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:53275/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@2187f3
[junit4:junit4] 2> 3373 T1132 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 3373 T1160 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:53275. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 3373 T1160 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:53275, initiating session
[junit4:junit4] 2> 3373 T1134 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:43974
[junit4:junit4] 2> 3374 T1134 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:43974
[junit4:junit4] 2> 3374 T1136 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d329a25f00004 with negotiated timeout 10000 for client /127.0.0.1:43974
[junit4:junit4] 2> 3374 T1160 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:53275, sessionid = 0x13d329a25f00004, negotiated timeout = 10000
[junit4:junit4] 2> 3375 T1161 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2187f3 name:ZooKeeperConnection Watcher:127.0.0.1:53275/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 3375 T1132 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 3376 T1132 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 3377 T1132 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4] 2> 3438 T1132 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4] 2> 3440 T1132 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:58158
[junit4:junit4] 2> 3440 T1132 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 3441 T1132 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 3441 T1132 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1362353599259
[junit4:junit4] 2> 3441 T1132 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1362353599259/solr.xml
[junit4:junit4] 2> 3442 T1132 oasc.CoreContainer.<init> New CoreContainer 7335748
[junit4:junit4] 2> 3442 T1132 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1362353599259/'
[junit4:junit4] 2> 3442 T1132 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1362353599259/'
[junit4:junit4] 2> 3461 T1132 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 3461 T1132 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 3461 T1132 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 3462 T1132 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 3462 T1132 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 3462 T1132 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 3463 T1132 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 3463 T1132 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 3463 T1132 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 3464 T1132 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 3471 T1132 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4] 2> 3490 T1132 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:53275/solr
[junit4:junit4] 2> 3490 T1132 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 3491 T1132 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:53275 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@1345f8c
[junit4:junit4] 2> 3492 T1132 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 3492 T1171 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:53275. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 3492 T1171 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:53275, initiating session
[junit4:junit4] 2> 3493 T1134 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:43975
[junit4:junit4] 2> 3493 T1134 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:43975
[junit4:junit4] 2> 3494 T1136 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d329a25f00005 with negotiated timeout 20000 for client /127.0.0.1:43975
[junit4:junit4] 2> 3494 T1171 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:53275, sessionid = 0x13d329a25f00005, negotiated timeout = 20000
[junit4:junit4] 2> 3494 T1172 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1345f8c name:ZooKeeperConnection Watcher:127.0.0.1:53275 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 3494 T1132 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 3495 T1137 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d329a25f00005
[junit4:junit4] 2> 3496 T1134 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:43975 which had sessionid 0x13d329a25f00005
[junit4:junit4] 2> 3495 T1132 oaz.ZooKeeper.close Session: 0x13d329a25f00005 closed
[junit4:junit4] 2> 3496 T1132 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 3497 T1172 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 3499 T1132 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:53275/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@c336c0
[junit4:junit4] 2> 3500 T1173 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:53275. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 3499 T1132 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 3500 T1173 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:53275, initiating session
[junit4:junit4] 2> 3500 T1134 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:43976
[junit4:junit4] 2> 3500 T1134 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:43976
[junit4:junit4] 2> 3501 T1136 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d329a25f00006 with negotiated timeout 20000 for client /127.0.0.1:43976
[junit4:junit4] 2> 3501 T1173 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:53275, sessionid = 0x13d329a25f00006, negotiated timeout = 20000
[junit4:junit4] 2> 3501 T1174 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@c336c0 name:ZooKeeperConnection Watcher:127.0.0.1:53275/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 3502 T1132 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 3502 T1137 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d329a25f00006 type:create cxid:0x1 zxid:0x49 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 3503 T1137 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d329a25f00006 type:create cxid:0x2 zxid:0x4a txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 3504 T1132 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 4506 T1132 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:58158_ds_
[junit4:junit4] 2> 4507 T1137 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d329a25f00006 type:delete cxid:0x9 zxid:0x4b txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:58158_ds_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:58158_ds_
[junit4:junit4] 2> 4507 T1132 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:58158_ds_
[junit4:junit4] 2> 4510 T1174 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 4510 T1154 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 4511 T1154 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 4511 T1161 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 4515 T1175 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1362353599259/collection1
[junit4:junit4] 2> 4515 T1175 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 4516 T1175 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 4516 T1175 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 4517 T1175 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1362353599259/collection1/'
[junit4:junit4] 2> 4517 T1175 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1362353599259/collection1/lib/README' to classloader
[junit4:junit4] 2> 4518 T1175 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1362353599259/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 4543 T1175 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 4584 T1175 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 4585 T1175 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4] 2> 4589 T1175 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4] 2> 4848 T1155 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 4849 T1155 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:35276/ds_",
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "core_node_name":"1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "node_name":"127.0.0.1:35276_ds_"}
[junit4:junit4] 2> 4851 T1154 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 4852 T1161 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 4852 T1174 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 4995 T1175 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 5001 T1175 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 5004 T1175 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 5012 T1175 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 5015 T1175 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 5017 T1175 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 5018 T1175 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 5018 T1175 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 5018 T1175 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 5019 T1175 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 5019 T1175 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 5020 T1175 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1362353599259/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty1/
[junit4:junit4] 2> 5020 T1175 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@64b87b
[junit4:junit4] 2> 5020 T1175 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 5020 T1175 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty1 forceNew:false
[junit4:junit4] 2> 5021 T1175 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty1
[junit4:junit4] 2> 5021 T1175 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty1/index/
[junit4:junit4] 2> 5021 T1175 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 5021 T1175 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty1/index forceNew:false
[junit4:junit4] 2> 5022 T1175 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(NRTCachingDirectory(org.apache.lucene.store.RAMDirectory@46ba6b lockFactory=org.apache.lucene.store.NativeFSLockFactory@227076; maxCacheMB=0.4580078125 maxMergeSizeMB=0.19140625)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> 5023 T1175 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4] 2> 5023 T1175 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty1/index
[junit4:junit4] 2> 5024 T1175 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 5024 T1175 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 5025 T1175 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 5025 T1175 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 5026 T1175 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 5026 T1175 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 5026 T1175 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 5026 T1175 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 5027 T1175 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 5029 T1175 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 5031 T1175 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty1
[junit4:junit4] 2> 5031 T1175 oass.SolrIndexSearcher.<init> Opening Searcher@465f19 main
[junit4:junit4] 2> 5032 T1175 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty1/tlog
[junit4:junit4] 2> 5032 T1175 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 5032 T1175 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 5036 T1176 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@465f19 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 5037 T1175 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 5037 T1175 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 6355 T1155 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 6356 T1155 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:58158/ds_",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "core_node_name":"2",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "node_name":"127.0.0.1:58158_ds_"}
[junit4:junit4] 2> 6356 T1155 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 2
[junit4:junit4] 2> 6356 T1155 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4:junit4] 2> 6360 T1161 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 6360 T1154 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 6360 T1174 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 7039 T1175 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 7039 T1175 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:58158/ds_ collection:collection1 shard:shard2
[junit4:junit4] 2> 7040 T1175 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4:junit4] 2> 7044 T1137 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d329a25f00006 type:delete cxid:0x46 zxid:0x5b txntype:-1 reqpath:n/a Error Path:/solr/collections/collection1/leaders Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders
[junit4:junit4] 2> 7045 T1175 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 7045 T1137 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d329a25f00006 type:create cxid:0x47 zxid:0x5c txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 7046 T1175 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 7047 T1175 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 7047 T1175 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:58158/ds_/collection1/
[junit4:junit4] 2> 7047 T1175 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 7047 T1175 oasc.SyncStrategy.syncToMe http://127.0.0.1:58158/ds_/collection1/ has no replicas
[junit4:junit4] 2> 7047 T1175 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:58158/ds_/collection1/
[junit4:junit4] 2> 7047 T1175 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4:junit4] 2> 7050 T1137 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d329a25f00006 type:create cxid:0x51 zxid:0x60 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 7863 T1155 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 7869 T1161 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 7869 T1174 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 7869 T1154 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 7903 T1175 oasc.ZkController.register We are http://127.0.0.1:58158/ds_/collection1/ and leader is http://127.0.0.1:58158/ds_/collection1/
[junit4:junit4] 2> 7903 T1175 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:58158/ds_
[junit4:junit4] 2> 7903 T1175 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 7904 T1175 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 7904 T1175 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 7905 T1175 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 7906 T1132 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
[junit4:junit4] 2> 7907 T1132 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 7907 T1132 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 7970 T1132 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4] 2> 7972 T1132 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:49875
[junit4:junit4] 2> 7972 T1132 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 7973 T1132 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 7973 T1132 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1362353603792
[junit4:junit4] 2> 7974 T1132 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1362353603792/solr.xml
[junit4:junit4] 2> 7974 T1132 oasc.CoreContainer.<init> New CoreContainer 9096842
[junit4:junit4] 2> 7974 T1132 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1362353603792/'
[junit4:junit4] 2> 7975 T1132 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1362353603792/'
[junit4:junit4] 2> 7999 T1132 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 8000 T1132 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 8000 T1132 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 8000 T1132 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 8001 T1132 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 8001 T1132 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 8001 T1132 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 8002 T1132 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 8002 T1132 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 8002 T1132 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 8009 T1132 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4] 2> 8021 T1132 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:53275/solr
[junit4:junit4] 2> 8022 T1132 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 8022 T1132 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:53275 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@91fade
[junit4:junit4] 2> 8023 T1132 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 8023 T1187 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:53275. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 8024 T1187 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:53275, initiating session
[junit4:junit4] 2> 8024 T1134 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:43979
[junit4:junit4] 2> 8024 T1134 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:43979
[junit4:junit4] 2> 8025 T1136 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d329a25f00007 with negotiated timeout 20000 for client /127.0.0.1:43979
[junit4:junit4] 2> 8025 T1187 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:53275, sessionid = 0x13d329a25f00007, negotiated timeout = 20000
[junit4:junit4] 2> 8025 T1188 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@91fade name:ZooKeeperConnection Watcher:127.0.0.1:53275 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 8026 T1132 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 8027 T1137 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d329a25f00007
[junit4:junit4] 2> 8027 T1188 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 8027 T1134 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:43979 which had sessionid 0x13d329a25f00007
[junit4:junit4] 2> 8027 T1132 oaz.ZooKeeper.close Session: 0x13d329a25f00007 closed
[junit4:junit4] 2> 8028 T1132 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 8030 T1132 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:53275/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@10122f8
[junit4:junit4] 2> 8031 T1189 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:53275. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 8031 T1132 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 8031 T1189 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:53275, initiating session
[junit4:junit4] 2> 8031 T1134 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:43980
[junit4:junit4] 2> 8032 T1134 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:43980
[junit4:junit4] 2> 8032 T1136 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d329a25f00008 with negotiated timeout 20000 for client /127.0.0.1:43980
[junit4:junit4] 2> 8032 T1189 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:53275, sessionid = 0x13d329a25f00008, negotiated timeout = 20000
[junit4:junit4] 2> 8033 T1190 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@10122f8 name:ZooKeeperConnection Watcher:127.0.0.1:53275/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 8033 T1132 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 8034 T1137 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d329a25f00008 type:create cxid:0x1 zxid:0x6d txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 8034 T1137 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d329a25f00008 type:create cxid:0x2 zxid:0x6e txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 8035 T1132 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 9037 T1132 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:49875_ds_
[junit4:junit4] 2> 9038 T1137 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d329a25f00008 type:delete cxid:0x9 zxid:0x6f txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:49875_ds_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:49875_ds_
[junit4:junit4] 2> 9038 T1132 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:49875_ds_
[junit4:junit4] 2> 9040 T1161 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 9041 T1154 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 9041 T1174 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 9041 T1154 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 9041 T1190 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 9041 T1174 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 9042 T1161 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 9046 T1191 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1362353603792/collection1
[junit4:junit4] 2> 9046 T1191 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 9046 T1191 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 9047 T1191 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 9048 T1191 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1362353603792/collection1/'
[junit4:junit4] 2> 9048 T1191 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1362353603792/collection1/lib/README' to classloader
[junit4:junit4] 2> 9048 T1191 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1362353603792/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 9074 T1191 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 9116 T1191 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 9117 T1191 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4] 2> 9121 T1191 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4] 2> 9373 T1155 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 9374 T1155 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:58158/ds_",
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "core_node_name":"2",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "node_name":"127.0.0.1:58158_ds_"}
[junit4:junit4] 2> 9377 T1154 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 9386 T1190 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 9386 T1161 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 9386 T1174 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 9574 T1191 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 9580 T1191 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 9582 T1191 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 9594 T1191 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 9597 T1191 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 9600 T1191 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 9600 T1191 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 9601 T1191 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 9601 T1191 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 9602 T1191 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 9602 T1191 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 9602 T1191 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1362353603792/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty2/
[junit4:junit4] 2> 9603 T1191 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@64b87b
[junit4:junit4] 2> 9603 T1191 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 9603 T1191 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty2 forceNew:false
[junit4:junit4] 2> 9604 T1191 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty2
[junit4:junit4] 2> 9604 T1191 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty2/index/
[junit4:junit4] 2> 9604 T1191 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty2/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 9604 T1191 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty2/index forceNew:false
[junit4:junit4] 2> 9605 T1191 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(NRTCachingDirectory(org.apache.lucene.store.RAMDirectory@fcbcdc lockFactory=org.apache.lucene.store.NativeFSLockFactory@1bcbfb5; maxCacheMB=0.4580078125 maxMergeSizeMB=0.19140625)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> 9606 T1191 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4] 2> 9606 T1191 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty2/index
[junit4:junit4] 2> 9607 T1191 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 9607 T1191 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 9608 T1191 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 9608 T1191 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 9608 T1191 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 9609 T1191 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 9609 T1191 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 9609 T1191 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 9610 T1191 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 9612 T1191 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 9614 T1191 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty2
[junit4:junit4] 2> 9615 T1191 oass.SolrIndexSearcher.<init> Opening Searcher@1735c9e main
[junit4:junit4] 2> 9615 T1191 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty2/tlog
[junit4:junit4] 2> 9615 T1191 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 9616 T1191 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 9619 T1192 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1735c9e main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 9621 T1191 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 9621 T1191 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 10890 T1155 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 10891 T1155 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49875/ds_",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "core_node_name":"3",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "node_name":"127.0.0.1:49875_ds_"}
[junit4:junit4] 2> 10891 T1155 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4] 2> 10891 T1155 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4] 2> 10895 T1190 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 10895 T1154 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 10895 T1161 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 10895 T1174 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 11622 T1191 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 11622 T1191 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:49875/ds_ collection:collection1 shard:shard1
[junit4:junit4] 2> 11623 T1191 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4] 2> 11627 T1137 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d329a25f00008 type:delete cxid:0x45 zxid:0x7e txntype:-1 reqpath:n/a Error Path:/solr/collections/collection1/leaders/shard1 Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders/shard1
[junit4:junit4] 2> 11628 T1191 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 11628 T1137 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d329a25f00008 type:create cxid:0x46 zxid:0x7f txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 11630 T1191 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 11630 T1191 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 11630 T1191 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:49875/ds_/collection1/
[junit4:junit4] 2> 11630 T1191 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 11630 T1191 oasc.SyncStrategy.syncToMe http://127.0.0.1:49875/ds_/collection1/ has no replicas
[junit4:junit4] 2> 11631 T1191 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:49875/ds_/collection1/
[junit4:junit4] 2> 11631 T1191 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4] 2> 11633 T1137 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d329a25f00008 type:create cxid:0x4f zxid:0x82 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 12399 T1155 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 12406 T1190 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 12406 T1161 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 12406 T1154 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 12406 T1174 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 12436 T1191 oasc.ZkController.register We are http://127.0.0.1:49875/ds_/collection1/ and leader is http://127.0.0.1:49875/ds_/collection1/
[junit4:junit4] 2> 12437 T1191 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:49875/ds_
[junit4:junit4] 2> 12437 T1191 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 12437 T1191 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 12437 T1191 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 12439 T1191 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 12440 T1132 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
[junit4:junit4] 2> 12440 T1132 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 12441 T1132 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 12504 T1132 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4] 2> 12513 T1132 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:59775
[junit4:junit4] 2> 12514 T1132 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 12514 T1132 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 12515 T1132 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1362353608325
[junit4:junit4] 2> 12515 T1132 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1362353608325/solr.xml
[junit4:junit4] 2> 12515 T1132 oasc.CoreContainer.<init> New CoreContainer 3438750
[junit4:junit4] 2> 12516 T1132 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1362353608325/'
[junit4:junit4] 2> 12516 T1132 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1362353608325/'
[junit4:junit4] 2> 12533 T1132 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 12534 T1132 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 12534 T1132 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 12534 T1132 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 12535 T1132 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 12535 T1132 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 12535 T1132 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 12536 T1132 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 12536 T1132 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 12536 T1132 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 12543 T1132 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4] 2> 12554 T1132 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:53275/solr
[junit4:junit4] 2> 12554 T1132 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 12555 T1132 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:53275 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@5762a9
[junit4:junit4] 2> 12556 T1132 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 12556 T1203 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:53275. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 12557 T1203 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:53275, initiating session
[junit4:junit4] 2> 12557 T1134 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:43988
[junit4:junit4] 2> 12557 T1134 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:43988
[junit4:junit4] 2> 12558 T1136 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d329a25f00009 with negotiated timeout 20000 for client /127.0.0.1:43988
[junit4:junit4] 2> 12558 T1203 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:53275, sessionid = 0x13d329a25f00009, negotiated timeout = 20000
[junit4:junit4] 2> 12558 T1204 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5762a9 name:ZooKeeperConnection Watcher:127.0.0.1:53275 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 12558 T1132 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 12559 T1137 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d329a25f00009
[junit4:junit4] 2> 12560 T1204 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 12560 T1134 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:43988 which had sessionid 0x13d329a25f00009
[junit4:junit4] 2> 12560 T1132 oaz.ZooKeeper.close Session: 0x13d329a25f00009 closed
[junit4:junit4] 2> 12560 T1132 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 12563 T1132 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:53275/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@52281e
[junit4:junit4] 2> 12564 T1205 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:53275. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 12564 T1132 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 12564 T1134 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:43989
[junit4:junit4] 2> 12564 T1205 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:53275, initiating session
[junit4:junit4] 2> 12565 T1134 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:43989
[junit4:junit4] 2> 12565 T1136 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d329a25f0000a with negotiated timeout 20000 for client /127.0.0.1:43989
[junit4:junit4] 2> 12565 T1205 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:53275, sessionid = 0x13d329a25f0000a, negotiated timeout = 20000
[junit4:junit4] 2> 12566 T1206 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@52281e name:ZooKeeperConnection Watcher:127.0.0.1:53275/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 12566 T1132 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 12567 T1137 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d329a25f0000a type:create cxid:0x1 zxid:0x8f txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 12567 T1137 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d329a25f0000a type:create cxid:0x2 zxid:0x90 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 12568 T1132 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 13571 T1132 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:59775_ds_
[junit4:junit4] 2> 13571 T1137 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d329a25f0000a type:delete cxid:0x9 zxid:0x91 txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:59775_ds_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:59775_ds_
[junit4:junit4] 2> 13572 T1132 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:59775_ds_
[junit4:junit4] 2> 13574 T1174 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 13575 T1154 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 13575 T1190 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 13575 T1161 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 13575 T1154 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 13575 T1190 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 13576 T1161 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 13576 T1206 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 13577 T1174 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 13582 T1207 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1362353608325/collection1
[junit4:junit4] 2> 13582 T1207 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 13583 T1207 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 13583 T1207 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 13584 T1207 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1362353608325/collection1/'
[junit4:junit4] 2> 13584 T1207 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1362353608325/collection1/lib/README' to classloader
[junit4:junit4] 2> 13584 T1207 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1362353608325/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 13610 T1207 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 13651 T1207 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 13652 T1207 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4] 2> 13656 T1207 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4] 2> 13911 T1155 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 13912 T1155 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49875/ds_",
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "core_node_name":"3",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "node_name":"127.0.0.1:49875_ds_"}
[junit4:junit4] 2> 13914 T1154 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 13915 T1174 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 13915 T1161 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 13915 T1206 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 13915 T1190 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 14064 T1207 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 14070 T1207 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 14073 T1207 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 14082 T1207 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 14084 T1207 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 14087 T1207 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 14088 T1207 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 14088 T1207 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 14089 T1207 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 14089 T1207 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 14090 T1207 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 14090 T1207 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1362353608325/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty3/
[junit4:junit4] 2> 14090 T1207 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@64b87b
[junit4:junit4] 2> 14090 T1207 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 14091 T1207 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty3 forceNew:false
[junit4:junit4] 2> 14091 T1207 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty3
[junit4:junit4] 2> 14091 T1207 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty3/index/
[junit4:junit4] 2> 14091 T1207 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty3/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 14092 T1207 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty3/index forceNew:false
[junit4:junit4] 2> 14093 T1207 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(NRTCachingDirectory(org.apache.lucene.store.RAMDirectory@1b71845 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1bd6084; maxCacheMB=0.4580078125 maxMergeSizeMB=0.19140625)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> 14093 T1207 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4] 2> 14093 T1207 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty3/index
[junit4:junit4] 2> 14094 T1207 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 14095 T1207 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 14095 T1207 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 14096 T1207 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 14096 T1207 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 14096 T1207 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 14096 T1207 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 14097 T1207 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 14097 T1207 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 14099 T1207 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 14102 T1207 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty3
[junit4:junit4] 2> 14102 T1207 oass.SolrIndexSearcher.<init> Opening Searcher@1e91dfe main
[junit4:junit4] 2> 14102 T1207 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty3/tlog
[junit4:junit4] 2> 14103 T1207 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 14103 T1207 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 14108 T1208 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1e91dfe main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 14109 T1207 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 14110 T1207 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 15419 T1155 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 15420 T1155 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:59775/ds_",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "core_node_name":"4",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "node_name":"127.0.0.1:59775_ds_"}
[junit4:junit4] 2> 15420 T1155 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4] 2> 15420 T1155 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4:junit4] 2> 15424 T1161 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 15424 T1154 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 15424 T1206 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 15424 T1174 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 15424 T1190 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 16111 T1207 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 16111 T1207 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:59775/ds_ collection:collection1 shard:shard2
[junit4:junit4] 2> 16114 T1207 oasc.ZkController.register We are http://127.0.0.1:59775/ds_/collection1/ and leader is http://127.0.0.1:58158/ds_/collection1/
[junit4:junit4] 2> 16114 T1207 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:59775/ds_
[junit4:junit4] 2> 16115 T1207 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 16115 T1207 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C55 name=collection1 org.apache.solr.core.SolrCore@b9d6ee url=http://127.0.0.1:59775/ds_/collection1 node=127.0.0.1:59775_ds_ C55_STATE=coll:collection1 core:collection1 props:{collection=collection1, core=collection1, base_url=http://127.0.0.1:59775/ds_, state=down, node_name=127.0.0.1:59775_ds_}
[junit4:junit4] 2> 16115 T1209 C55 P59775 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 16115 T1207 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 16116 T1209 C55 P59775 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 16116 T1209 C55 P59775 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 16116 T1209 C55 P59775 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 16117 T1132 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
[junit4:junit4] 2> 16117 T1209 C55 P59775 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 16117 T1132 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 16118 T1132 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 16120 T1132 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 16121 T1132 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 16125 T1170 oasc.CoreContainer.create Creating SolrCore 'onenodecollectioncore' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1362353599259/onenodecollectioncore
[junit4:junit4] 2> 16125 T1170 oasc.ZkController.createCollectionZkNode Check for collection zkNode:onenodecollection
[junit4:junit4] 2> 16126 T1170 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:onenodecollection
[junit4:junit4] 2> 16127 T1170 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4] 2> 16128 T1170 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
[junit4:junit4] 2> 16128 T1170 oascc.SolrZkClient.makePath makePath: /collections/onenodecollection
[junit4:junit4] 2> 16130 T1170 oasc.ZkController.readConfigName Load collection config from:/collections/onenodecollection
[junit4:junit4] 2> 16131 T1170 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1362353599259/onenodecollectioncore/'
[junit4:junit4] 2> 16183 T1170 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 16228 T1170 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 16229 T1170 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4] 2> 16240 T1170 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4] 2> 16645 T1170 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 16658 T1170 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 16661 T1170 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 16670 T1170 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 16673 T1170 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 16676 T1170 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 16677 T1170 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 16677 T1170 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 16677 T1170 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 16678 T1170 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 16679 T1170 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 16679 T1170 oasc.SolrCore.<init> [onenodecollectioncore] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1362353599259/onenodecollectioncore/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1362353595885/onenodecollection/
[junit4:junit4] 2> 16679 T1170 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@64b87b
[junit4:junit4] 2> 16680 T1170 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 16680 T1170 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1362353595885/onenodecollection forceNew:false
[junit4:junit4] 2> 16681 T1170 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1362353595885/onenodecollection
[junit4:junit4] 2> 16681 T1170 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1362353595885/onenodecollection/index/
[junit4:junit4] 2> 16681 T1170 oasc.SolrCore.initIndex WARNING [onenodecollectioncore] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1362353595885/onenodecollection/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 16682 T1170 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1362353595885/onenodecollection/index forceNew:false
[junit4:junit4] 2> 16683 T1170 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(NRTCachingDirectory(org.apache.lucene.store.RAMDirectory@17e95db lockFactory=org.apache.lucene.store.NativeFSLockFactory@1b4d3a1; maxCacheMB=0.4580078125 maxMergeSizeMB=0.19140625)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> 16683 T1170 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4] 2> 16684 T1170 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1362353595885/onenodecollection/index
[junit4:junit4] 2> 16685 T1170 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 16685 T1170 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 16685 T1170 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 16686 T1170 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 16686 T1170 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 16687 T1170 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 16687 T1170 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 16687 T1170 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 16688 T1170 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 16690 T1170 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 16692 T1170 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1362353595885/onenodecollection
[junit4:junit4] 2> 16693 T1170 oass.SolrIndexSearcher.<init> Opening Searcher@13a9497 main
[junit4:junit4] 2> 16693 T1170 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1362353595885/onenodecollection/tlog
[junit4:junit4] 2> 16694 T1170 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 16694 T1170 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 16697 T1170 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1362353595885/onenodecollection
[junit4:junit4] 2> 16699 T1211 oasc.SolrCore.registerSearcher [onenodecollectioncore] Registered new searcher Searcher@13a9497 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 16700 T1170 oasc.ZkController.publish publishing core=onenodecollectioncore state=down
[junit4:junit4] 2> 16927 T1155 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 16928 T1155 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:59775/ds_",
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "core_node_name":"4",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "node_name":"127.0.0.1:59775_ds_"}
[junit4:junit4] 2> 16931 T1155 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "roles":"none",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "collection":"onenodecollection",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "core":"onenodecollectioncore",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:58158/ds_",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "node_name":"127.0.0.1:58158_ds_"}
[junit4:junit4] 2> 16931 T1155 oasc.Overseer$ClusterStateUpdater.createCollection Create collection onenodecollection with numShards 1
[junit4:junit4] 2> 16931 T1155 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4] 2> 17156 T1174 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 17156 T1190 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 17156 T1161 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 17156 T1154 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 17156 T1206 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 17702 T1170 oasc.CoreContainer.registerCore registering core: onenodecollectioncore
[junit4:junit4] 2> 17703 T1170 oasc.ZkController.register Register replica - core:onenodecollectioncore address:http://127.0.0.1:58158/ds_ collection:onenodecollection shard:shard1
[junit4:junit4] 2> 17704 T1170 oascc.SolrZkClient.makePath makePath: /collections/onenodecollection/leader_elect/shard1/election
[junit4:junit4] 2> 17709 T1137 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d329a25f00006 type:delete cxid:0x9f zxid:0xac txntype:-1 reqpath:n/a Error Path:/solr/collections/onenodecollection/leaders Error:KeeperErrorCode = NoNode for /solr/collections/onenodecollection/leaders
[junit4:junit4] 2> 17710 T1170 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 17710 T1137 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d329a25f00006 type:create cxid:0xa0 zxid:0xad txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 17711 T1170 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 17712 T1170 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 17712 T1170 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:58158/ds_/onenodecollectioncore/
[junit4:junit4] 2> 17712 T1170 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 17713 T1170 oasc.SyncStrategy.syncToMe http://127.0.0.1:58158/ds_/onenodecollectioncore/ has no replicas
[junit4:junit4] 2> 17713 T1170 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:58158/ds_/onenodecollectioncore/
[junit4:junit4] 2> 17713 T1170 oascc.SolrZkClient.makePath makePath: /collections/onenodecollection/leaders/shard1
[junit4:junit4] 2> 17717 T1137 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d329a25f00006 type:create cxid:0xaa zxid:0xb1 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 18661 T1155 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 18666 T1161 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 18666 T1174 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 18666 T1206 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 18666 T1154 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 18666 T1190 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 18671 T1170 oasc.ZkController.register We are http://127.0.0.1:58158/ds_/onenodecollectioncore/ and leader is http://127.0.0.1:58158/ds_/onenodecollectioncore/
[junit4:junit4] 2> 18672 T1170 oasc.ZkController.register No LogReplay needed for core=onenodecollectioncore baseURL=http://127.0.0.1:58158/ds_
[junit4:junit4] 2> 18672 T1170 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 18673 T1170 oasc.ZkController.publish publishing core=onenodecollectioncore state=active
[junit4:junit4] 2> 18674 T1170 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 18675 T1132 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: onenodecollection failOnTimeout:true timeout (sec):230
[junit4:junit4] 2> 18676 T1132 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 19678 T1132 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> C55_STATE=coll:collection1 core:collection1 props:{collection=collection1, shard=shard2, core=collection1, base_url=http://127.0.0.1:59775/ds_, state=recovering, node_name=127.0.0.1:59775_ds_}
[junit4:junit4] 2> 20121 T1209 C55 P59775 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:58158/ds_/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 20122 T1209 C55 P59775 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:59775/ds_ START replicas=[http://127.0.0.1:58158/ds_/collection1/] nUpdates=100
[junit4:junit4] 2> 20122 T1209 C55 P59775 oasu.PeerSync.sync WARNING no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 20123 T1209 C55 P59775 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 20123 T1209 C55 P59775 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 20123 T1209 C55 P59775 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 20123 T1209 C55 P59775 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 20123 T1209 C55 P59775 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:58158/ds_/collection1/. core=collection1
[junit4:junit4] 2> 20124 T1209 C55 P59775 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> ASYNC NEW_CORE C56 name=collection1 org.apache.solr.core.SolrCore@1f8b671 url=http://127.0.0.1:58158/ds_/collection1 node=127.0.0.1:58158_ds_ C56_STATE=coll:collection1 core:collection1 props:{collection=collection1, shard=shard2, core=collection1, base_url=http://127.0.0.1:58158/ds_, state=active, node_name=127.0.0.1:58158_ds_, leader=true}
[junit4:junit4] 2> 20127 T1168 C56 P58158 REQ /get {version=2&distrib=false&getVersions=100&wt=javabin&qt=/get} status=0 QTime=0
[junit4:junit4] 2> 20130 T1170 C56 P58158 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 20130 T1170 C56 P58158 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty1
[junit4:junit4] 2> 20131 T1170 C56 P58158 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(NRTCachingDirectory(org.apache.lucene.store.RAMDirectory@46ba6b lockFactory=org.apache.lucene.store.NativeFSLockFactory@227076; maxCacheMB=0.4580078125 maxMergeSizeMB=0.19140625)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> 20132 T1170 C56 P58158 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4] 2> 20133 T1170 C56 P58158 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(NRTCachingDirectory(org.apache.lucene.store.RAMDirectory@46ba6b lockFactory=org.apache.lucene.store.NativeFSLockFactory@227076; maxCacheMB=0.4580078125 maxMergeSizeMB=0.19140625)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> commit{dir=MockDirWrapper(NRTCachingDirectory(org.apache.lucene.store.RAMDirectory@46ba6b lockFactory=org.apache.lucene.store.NativeFSLockFactory@227076; maxCacheMB=0.4580078125 maxMergeSizeMB=0.19140625)),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4] 2> 20133 T1170 C56 P58158 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4] 2> 20133 T1170 C56 P58158 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty1
[junit4:junit4] 2> 20134 T1170 C56 P58158 oass.SolrIndexSearcher.<init> Opening Searcher@bdb4bc realtime
[junit4:junit4] 2> 20134 T1170 C56 P58158 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 20134 T1170 C56 P58158 /update {version=2&softCommit=false&commit=true&wt=javabin&waitSearcher=true&openSearcher=false&commit_end_point=true} {commit=} 0 4
[junit4:junit4] 2> 20135 T1209 C55 P59775 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 20135 T1209 C55 P59775 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 20142 T1168 C56 P58158 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 20143 T1168 C56 P58158 REQ /replication {version=2&command=indexversion&wt=javabin&qt=/replication} status=0 QTime=7
[junit4:junit4] 2> 20143 T1209 C55 P59775 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4] 2> 20144 T1209 C55 P59775 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 20144 T1209 C55 P59775 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 20146 T1167 C56 P58158 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty1
[junit4:junit4] 2> 20146 T1167 C56 P58158 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty1/index
[junit4:junit4] 2> 20146 T1167 C56 P58158 REQ /replication {version=2&command=filelist&generation=2&wt=javabin&qt=/replication} status=0 QTime=0
[junit4:junit4] 2> 20147 T1209 C55 P59775 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4] 2> 20148 T1209 C55 P59775 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty3/index.20130303153336029 forceNew:false
[junit4:junit4] 2> 20148 T1209 C55 P59775 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty3
[junit4:junit4] 2> 20149 T1209 C55 P59775 oash.SnapPuller.fetchLatestIndex Starting download to MockDirWrapper(org.apache.lucene.store.RAMDirectory@d8e0a7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7fe0e8) fullCopy=false
[junit4:junit4] 2> 20151 T1170 C56 P58158 REQ /replication {file=segments_2&command=filecontent&generation=2&wt=filestream&checksum=true&qt=/replication} status=0 QTime=0
[junit4:junit4] 2> 20152 T1209 C55 P59775 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty3/index
[junit4:junit4] 2> 20152 T1209 C55 P59775 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 20153 T1209 C55 P59775 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty3
[junit4:junit4] 2> 20153 T1209 C55 P59775 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty3
[junit4:junit4] 2> 20154 T1209 C55 P59775 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 20154 T1209 C55 P59775 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4] 2> 20154 T1209 C55 P59775 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty3
[junit4:junit4] 2> 20155 T1209 C55 P59775 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(NRTCachingDirectory(org.apache.lucene.store.RAMDirectory@1b71845 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1bd6084; maxCacheMB=0.4580078125 maxMergeSizeMB=0.19140625)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> commit{dir=MockDirWrapper(NRTCachingDirectory(org.apache.lucene.store.RAMDirectory@1b71845 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1bd6084; maxCacheMB=0.4580078125 maxMergeSizeMB=0.19140625)),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4] 2> 20156 T1209 C55 P59775 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4] 2> 20156 T1209 C55 P59775 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 20156 T1209 C55 P59775 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty3
[junit4:junit4] 2> 20157 T1209 C55 P59775 oass.SolrIndexSearcher.<init> Opening Searcher@3a9b12 main
[junit4:junit4] 2> 20157 T1208 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3a9b12 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 20158 T1208 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty3/index
[junit4:junit4] 2> 20158 T1209 C55 P59775 oash.SnapPuller.fetchLatestIndex removing temporary index download directory files MockDirWrapper(org.apache.lucene.store.RAMDirectory@d8e0a7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7fe0e8)
[junit4:junit4] 2> 20158 T1209 C55 P59775 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty3/index.20130303153336029
[junit4:junit4] 2> 20159 T1209 C55 P59775 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty3/index
[junit4:junit4] 2> 20159 T1209 C55 P59775 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 20159 T1209 C55 P59775 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 20159 T1209 C55 P59775 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 20159 T1209 C55 P59775 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 20161 T1209 C55 P59775 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 20170 T1155 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 20171 T1155 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "roles":"none",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "collection":"onenodecollection",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "core":"onenodecollectioncore",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:58158/ds_",
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "node_name":"127.0.0.1:58158_ds_"}
[junit4:junit4] 2> 20174 T1155 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:59775/ds_",
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "core_node_name":"4",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "node_name":"127.0.0.1:59775_ds_"}
[junit4:junit4] 2> 20256 T1174 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 20256 T1206 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 20256 T1190 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 20256 T1161 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 20256 T1154 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 20679 T1132 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 20680 T1132 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: onenodecollection
[junit4:junit4] 2> 20681 T1132 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 20708 T1132 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:53275 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@18d19bc
[junit4:junit4] 2> 20709 T1132 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 20709 T1214 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:53275. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 20709 T1214 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:53275, initiating session
[junit4:junit4] 2> 20709 T1134 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:44008
[junit4:junit4] 2> 20710 T1134 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:44008
[junit4:junit4] 2> 20711 T1136 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d329a25f0000b with negotiated timeout 10000 for client /127.0.0.1:44008
[junit4:junit4] 2> 20711 T1214 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:53275, sessionid = 0x13d329a25f0000b, negotiated timeout = 10000
[junit4:junit4] 2> 20711 T1215 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@18d19bc name:ZooKeeperConnection Watcher:127.0.0.1:53275 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 20711 T1132 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 20740 T1137 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d329a25f0000b
[junit4:junit4] 2> 20741 T1215 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 20741 T1134 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:44008 which had sessionid 0x13d329a25f0000b
[junit4:junit4] 2> 20741 T1132 oaz.ZooKeeper.close Session: 0x13d329a25f0000b closed
[junit4:junit4] 2> 20741 T1137 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d329a25f00004
[junit4:junit4] 2> 20742 T1161 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 20742 T1132 oaz.ZooKeeper.close Session: 0x13d329a25f00004 closed
[junit4:junit4] 2> 20742 T1134 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:43974 which had sessionid 0x13d329a25f00004
[junit4:junit4] 2> 20744 T1132 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 35276
[junit4:junit4] 2> 20744 T1132 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=20508092
[junit4:junit4] 2> 21762 T1155 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 21762 T1155 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:35276/ds_",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "core_node_name":"1",
[junit4:junit4] 2> "node_name":"127.0.0.1:35276_ds_"}
[junit4:junit4] 2> 21765 T1154 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 21765 T1190 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 21765 T1206 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 21766 T1174 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 23747 T1132 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 23748 T1132 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 23768 T1132 oasc.SolrCore.close [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@d9acb9
[junit4:junit4] 2> 23771 T1132 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty3/index
[junit4:junit4] 2> 23772 T1132 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty3
[junit4:junit4] 2> 23773 T1132 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
[junit4:junit4] 2> 23773 T1132 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4:junit4] 2> 23773 T1132 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4] 2> 23774 T1132 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4:junit4] 2> 23774 T1132 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4:junit4] 2> 23775 T1132 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/control/data/index
[junit4:junit4] 2> 23775 T1132 oasc.CachingDirectoryFactory.close Closing directory when closing factory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/control/data
[junit4:junit4] 2> 23775 T1132 oasc.CachingDirectoryFactory.closeDirectory Closing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/control/data
[junit4:junit4] 2> 23776 T1132 oasc.CachingDirectoryFactory.close Closing directory when closing factory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/control/data/index
[junit4:junit4] 2> 23776 T1132 oasc.CachingDirectoryFactory.closeDirectory Closing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/control/data/index
[junit4:junit4] 2> 23777 T1155 oasc.Overseer$ClusterStateUpdater.amILeader According to ZK I (id=89283205260115971-127.0.0.1:35276_ds_-n_0000000000) am no longer a leader.
[junit4:junit4] 2> 23778 T1137 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d329a25f00003
[junit4:junit4] 2> 23779 T1206 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 23779 T1134 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:43971 which had sessionid 0x13d329a25f00003
[junit4:junit4] 2> 23779 T1190 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 23779 T1174 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 23780 T1190 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 23779 T1132 oaz.ZooKeeper.close Session: 0x13d329a25f00003 closed
[junit4:junit4] 2> 23780 T1174 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 23780 T1206 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 23781 T1137 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d329a25f00006 type:delete cxid:0xba zxid:0xcd txntype:-1 reqpath:n/a Error Path:/solr/overseer_elect/leader Error:KeeperErrorCode = NoNode for /solr/overseer_elect/leader
[junit4:junit4] 2> 23782 T1174 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4] 2> 23783 T1174 oasc.Overseer.start Overseer (id=89283205260115974-127.0.0.1:58158_ds_-n_0000000001) starting
[junit4:junit4] 2> 23784 T1137 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d329a25f00006 type:create cxid:0xbf zxid:0xcf txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 23784 T1137 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d329a25f00006 type:create cxid:0xc0 zxid:0xd0 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 23785 T1137 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d329a25f00006 type:create cxid:0xc1 zxid:0xd1 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 23786 T1137 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d329a25f00006 type:create cxid:0xc2 zxid:0xd2 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 23786 T1217 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4] 2> 23795 T1216 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4] 2> 23796 T1216 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 23796 T1216 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:35276/ds_",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "core_node_name":"1",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "node_name":"127.0.0.1:35276_ds_"}
[junit4:junit4] 2> 23799 T1206 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 23799 T1174 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 23799 T1190 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 23805 T1132 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/ds_,null}
[junit4:junit4] 2> 23856 T1132 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 58158
[junit4:junit4] 2> 23856 T1132 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=7335748
[junit4:junit4] 2> 25079 T1154 oascc.ZkStateReader$3.process WARNING ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 25079 T1154 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 25079 T1154 oascc.ZkStateReader$2.process WARNING ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 25079 T1154 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 25302 T1216 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 25302 T1216 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4:junit4] 2> "roles":"none",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "collection":"onenodecollection",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "core":"onenodecollectioncore",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:58158/ds_",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:58158_ds__onenodecollectioncore",
[junit4:junit4] 2> "node_name":"127.0.0.1:58158_ds_"}
[junit4:junit4] 2> 25305 T1216 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:58158/ds_",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "core_node_name":"2",
[junit4:junit4] 2> "node_name":"127.0.0.1:58158_ds_"}
[junit4:junit4] 2> 25307 T1174 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 25307 T1190 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 25307 T1206 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 26858 T1132 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 26859 T1132 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 26860 T1132 oasc.ZkController.publish publishing core=onenodecollectioncore state=down
[junit4:junit4] 2> 26862 T1132 oasc.SolrCore.close [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@1f8b671
[junit4:junit4] 2> 26865 T1132 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty3/index
[junit4:junit4] 2> 26866 T1132 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty3
[junit4:junit4] 2> 26866 T1132 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=1,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
[junit4:junit4] 2> 26867 T1132 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4:junit4] 2> 26867 T1132 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4] 2> 26867 T1132 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4:junit4] 2> C56_STATE=coll:collection1 core:collection1 props:{collection=collection1, shard=shard2, core=collection1, base_url=http://127.0.0.1:58158/ds_, state=down, node_name=127.0.0.1:58158_ds_, leader=true}
[junit4:junit4] 2> 26868 T1132 C56 P58158 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty1/index
[junit4:junit4] 2> 26869 T1132 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4:junit4] 2> 26869 T1132 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty1/index
[junit4:junit4] 2> 26869 T1132 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty1/index
[junit4:junit4] 2> 26870 T1132 oasc.CachingDirectoryFactory.close Closing directory when closing factory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty1/index
[junit4:junit4] 2> 26870 T1132 oasc.CachingDirectoryFactory.closeDirectory Closing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty1/index
[junit4:junit4] 2> 26870 T1132 oasc.CachingDirectoryFactory.close Closing directory when closing factory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty1
[junit4:junit4] 2> 26871 T1132 oasc.CachingDirectoryFactory.closeDirectory Closing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty1
[junit4:junit4] 2> 26871 T1132 oasc.SolrCore.close [onenodecollectioncore] CLOSING SolrCore org.apache.solr.core.SolrCore@1ab038c
[junit4:junit4] 2> 26875 T1132 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
[junit4:junit4] 2> 26876 T1132 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4:junit4] 2> 26876 T1132 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4] 2> 26876 T1132 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4:junit4] 2> 26877 T1132 oasc.SolrCore.closeSearcher [onenodecollectioncore] Closing main searcher on request.
[junit4:junit4] 2> 26877 T1132 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1362353595885/onenodecollection/index
[junit4:junit4] 2> 26878 T1132 oasc.CachingDirectoryFactory.close Closing directory when closing factory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1362353595885/onenodecollection
[junit4:junit4] 2> 26878 T1132 oasc.CachingDirectoryFactory.closeDirectory Closing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1362353595885/onenodecollection
[junit4:junit4] 2> 26878 T1132 oasc.CachingDirectoryFactory.close Closing directory when closing factory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1362353595885/onenodecollection/index
[junit4:junit4] 2> 26879 T1132 oasc.CachingDirectoryFactory.closeDirectory Closing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1362353595885/onenodecollection/index
[junit4:junit4] 2> 26879 T1216 oasc.Overseer$ClusterStateUpdater.amILeader According to ZK I (id=89283205260115974-127.0.0.1:58158_ds_-n_0000000001) am no longer a leader.
[junit4:junit4] 2> 26880 T1137 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d329a25f00006
[junit4:junit4] 2> 26881 T1134 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:43976 which had sessionid 0x13d329a25f00006
[junit4:junit4] 2> 26881 T1132 oaz.ZooKeeper.close Session: 0x13d329a25f00006 closed
[junit4:junit4] 2> 26881 T1137 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d329a25f0000a type:delete cxid:0x53 zxid:0xe3 txntype:-1 reqpath:n/a Error Path:/solr/collections/collection1/leaders/shard2 Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders/shard2
[junit4:junit4] 2> 26882 T1206 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 26882 T1137 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d329a25f00008 type:delete cxid:0x6e zxid:0xe4 txntype:-1 reqpath:n/a Error Path:/solr/overseer_elect/leader Error:KeeperErrorCode = NoNode for /solr/overseer_elect/leader
[junit4:junit4] 2> 26882 T1137 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d329a25f0000a type:create cxid:0x54 zxid:0xe5 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 26883 T1190 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4] 2> 26884 T1206 oasc.ShardLeaderElectionContext.shouldIBeLeader Checking if I should try and be the leader.
[junit4:junit4] 2> 26884 T1206 oasc.ShardLeaderElectionContext.shouldIBeLeader My last published State was Active, it's okay to be the leader.
[junit4:junit4] 2> 26884 T1206 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 26884 T1206 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:59775/ds_/collection1/
[junit4:junit4] 2> 26885 T1206 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:59775/ds_ START replicas=[http://127.0.0.1:58158/ds_/collection1/] nUpdates=100
[junit4:junit4] 2> 26885 T1206 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:59775/ds_ DONE. We have no versions. sync failed.
[junit4:junit4] 2> 26886 T1206 oasc.SyncStrategy.syncReplicas Leader's attempt to sync with shard failed, moving to the next canidate
[junit4:junit4] 2> 26886 T1206 oasc.ShardLeaderElectionContext.runLeaderProcess We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
[junit4:junit4] 2> 26886 T1206 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:59775/ds_/collection1/
[junit4:junit4] 2> 26886 T1206 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4:junit4] 2> 26888 T1190 oasc.Overseer.start Overseer (id=89283205260115976-127.0.0.1:49875_ds_-n_0000000002) starting
[junit4:junit4] 2> 26888 T1137 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d329a25f00008 type:create cxid:0x73 zxid:0xe8 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 26889 T1137 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d329a25f00008 type:create cxid:0x74 zxid:0xea txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 26890 T1137 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d329a25f0000a type:create cxid:0x5c zxid:0xeb txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 26890 T1137 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d329a25f00008 type:create cxid:0x75 zxid:0xec txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 26891 T1137 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d329a25f00008 type:create cxid:0x76 zxid:0xee txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 26891 T1206 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 26892 T1220 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4] 2> 26893 T1190 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 26893 T1190 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 26894 T1206 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 26894 T1219 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4] 2> 26895 T1219 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 26896 T1219 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:58158/ds_",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "core_node_name":"2",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "node_name":"127.0.0.1:58158_ds_"}
[junit4:junit4] 2> 26898 T1219 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "roles":"none",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "collection":"onenodecollection",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "core":"onenodecollectioncore",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:58158/ds_",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "node_name":"127.0.0.1:58158_ds_"}
[junit4:junit4] 2> 26903 T1132 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/ds_,null}
[junit4:junit4] 2> 26946 T1190 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 26946 T1206 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 26955 T1132 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 49875
[junit4:junit4] 2> 26955 T1132 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=9096842
[junit4:junit4] 2> 27956 T1132 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 27957 T1132 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 27959 T1132 oasc.SolrCore.close [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@1c3b982
[junit4:junit4] 2> 27962 T1132 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty3/index
[junit4:junit4] 2> 27963 T1132 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty3
[junit4:junit4] 2> 27963 T1132 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
[junit4:junit4] 2> 27964 T1132 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4:junit4] 2> 27964 T1132 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4] 2> 27964 T1132 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4:junit4] 2> 27965 T1132 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4:junit4] 2> 27965 T1132 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty2/index
[junit4:junit4] 2> 27966 T1132 oasc.CachingDirectoryFactory.close Closing directory when closing factory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty2/index
[junit4:junit4] 2> 27966 T1132 oasc.CachingDirectoryFactory.closeDirectory Closing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty2/index
[junit4:junit4] 2> 27966 T1132 oasc.CachingDirectoryFactory.close Closing directory when closing factory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty2
[junit4:junit4] 2> 27967 T1132 oasc.CachingDirectoryFactory.closeDirectory Closing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty2
[junit4:junit4] 2> 27967 T1219 oasc.Overseer$ClusterStateUpdater.amILeader According to ZK I (id=89283205260115976-127.0.0.1:49875_ds_-n_0000000002) am no longer a leader.
[junit4:junit4] 2> 27968 T1137 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d329a25f00008
[junit4:junit4] 2> 27969 T1206 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 27969 T1134 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:43980 which had sessionid 0x13d329a25f00008
[junit4:junit4] 2> 27969 T1132 oaz.ZooKeeper.close Session: 0x13d329a25f00008 closed
[junit4:junit4] 2> 27970 T1206 oascc.ZkStateReader$3.process Updating live nodes... (1)
[junit4:junit4] 2> 27971 T1137 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d329a25f0000a type:delete cxid:0x67 zxid:0xff txntype:-1 reqpath:n/a Error Path:/solr/overseer_elect/leader Error:KeeperErrorCode = NoNode for /solr/overseer_elect/leader
[junit4:junit4] 2> 27971 T1206 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4] 2> 27973 T1206 oasc.Overseer.start Overseer (id=89283205260115978-127.0.0.1:59775_ds_-n_0000000003) starting
[junit4:junit4] 2> 27973 T1137 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d329a25f0000a type:create cxid:0x6c zxid:0x101 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 27974 T1137 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d329a25f0000a type:create cxid:0x6d zxid:0x102 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 27974 T1137 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d329a25f0000a type:create cxid:0x6e zxid:0x103 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 27975 T1137 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d329a25f0000a type:create cxid:0x6f zxid:0x104 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 27975 T1222 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4] 2> 27976 T1221 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4] 2> 27978 T1221 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 27978 T1221 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49875/ds_",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "core_node_name":"3",
[junit4:junit4] 2> "node_name":"127.0.0.1:49875_ds_"}
[junit4:junit4] 2> 27981 T1221 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49875/ds_",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "core_node_name":"3",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "node_name":"127.0.0.1:49875_ds_"}
[junit4:junit4] 2> 27991 T1132 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/ds_,null}
[junit4:junit4] 2> 28029 T1206 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 28044 T1132 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 59775
[junit4:junit4] 2> 28044 T1132 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=3438750
[junit4:junit4] 2> 28181 T1174 oascc.ZkStateReader$3.process WARNING ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 28181 T1174 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 28181 T1174 oascc.ZkStateReader$2.process WARNING ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 28181 T1174 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 29045 T1132 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 29046 T1132 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 29049 T1132 oasc.SolrCore.close [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@b9d6ee
[junit4:junit4] 2> 29053 T1132 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty3/index
[junit4:junit4] 2> 29054 T1132 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty3
[junit4:junit4] 2> 29055 T1132 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty3/index
[junit4:junit4] 2> 29055 T1132 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty3
[junit4:junit4] 2> 29056 T1132 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
[junit4:junit4] 2> 29056 T1132 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4:junit4] 2> 29057 T1132 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4] 2> 29057 T1132 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4:junit4] 2> C55_STATE=coll:collection1 core:collection1 props:{collection=collection1, shard=shard2, core=collection1, base_url=http://127.0.0.1:59775/ds_, state=active, node_name=127.0.0.1:59775_ds_, leader=true}
[junit4:junit4] 2> 29058 T1132 C55 P59775 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty3/index
[junit4:junit4] 2> 29059 T1132 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4:junit4] 2> 29059 T1132 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty3/index
[junit4:junit4] 2> 29060 T1132 oasc.CachingDirectoryFactory.close Closing directory when closing factory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty3/index
[junit4:junit4] 2> 29061 T1132 oasc.CachingDirectoryFactory.closeDirectory Closing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty3/index
[junit4:junit4] 2> 29061 T1132 oasc.CachingDirectoryFactory.close Closing directory when closing factory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty3/index.20130303153336029
[junit4:junit4] 2> 29062 T1132 oasc.CachingDirectoryFactory.closeDirectory Closing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty3/index.20130303153336029
[junit4:junit4] 2> 29063 T1132 oasc.CachingDirectoryFactory.close Closing directory when closing factory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty3
[junit4:junit4] 2> 29063 T1132 oasc.CachingDirectoryFactory.closeDirectory Closing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362353595885/jetty3
[junit4:junit4] 2> 29066 T1221 oasc.Overseer$ClusterStateUpdater.amILeader According to ZK I (id=89283205260115978-127.0.0.1:59775_ds_-n_0000000003) am no longer a leader.
[junit4:junit4] 2> 29066 T1137 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d329a25f0000a
[junit4:junit4] 2> 29087 T1206 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 29087 T1134 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:43989 which had sessionid 0x13d329a25f0000a
[junit4:junit4] 2> 29087 T1132 oaz.ZooKeeper.close Session: 0x13d329a25f0000a closed
[junit4:junit4] 2> 29087 T1206 oascc.ZkStateReader$2.process WARNING ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 29088 T1206 oascc.ZkStateReader$3.process WARNING ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 29088 T1206 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 29110 T1132 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/ds_,null}
[junit4:junit4] 2> 29167 T1132 oas.SolrTestCaseJ4.tearDown ###Ending testDistribSearch
[junit4:junit4] 2> 29168 T1132 oazs.ZooKeeperServer.shutdown shutting down
[junit4:junit4] 2> 29168 T1132 oazs.SessionTrackerImpl.shutdown Shutting down
[junit4:junit4] 2> 29168 T1132 oazs.PrepRequestProcessor.shutdown Shutting down
[junit4:junit4] 2> 29169 T1137 oazs.PrepRequestProcessor.run PrepRequestProcessor exited loop!
[junit4:junit4] 2> 29169 T1132 oazs.SyncRequestProcessor.shutdown Shutting down
[junit4:junit4] 2> 29169 T1136 oazs.SyncRequestProcessor.run SyncRequestProcessor exited!
[junit4:junit4] 2> 29170 T1132 oazs.FinalRequestProcessor.shutdown shutdown of request processor complete
[junit4:junit4] 2> 29171 T1132 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:53275 53275
[junit4:junit4] 2> 29173 T1134 oazs.NIOServerCnxnFactory.run NIOServerCnxn factory exited run method
[junit4:junit4] 2> 29173 T1132 oazs.ZooKeeperServer.shutdown shutting down
[junit4:junit4] 2> 29174 T1132 oazs.SessionTrackerImpl.shutdown Shutting down
[junit4:junit4] 2> 29174 T1132 oazs.PrepRequestProcessor.shutdown Shutting down
[junit4:junit4] 2> 29174 T1132 oazs.SyncRequestProcessor.shutdown Shutting down
[junit4:junit4] 2> 29175 T1132 oazs.FinalRequestProcessor.shutdown shutdown of request processor complet
[...truncated too long message...]
Test(UnloadDistributedZkTest.java:77)
[junit4:junit4] > at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:806)
[junit4:junit4] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4:junit4] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
[junit4:junit4] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4:junit4] > at java.lang.reflect.Method.invoke(Method.java:487)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
[junit4:junit4] > at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
[junit4:junit4] > at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
[junit4:junit4] > at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
[junit4:junit4] > at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
[junit4:junit4] > at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
[junit4:junit4] > at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
[junit4:junit4] > at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
[junit4:junit4] > at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4] > at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
[junit4:junit4] > at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4:junit4] > at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
[junit4:junit4] > at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
[junit4:junit4] > at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] 2> 136346 T2567 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4:junit4] 2> 136357 T2566 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 3 leaked thread(s).
[junit4:junit4] 2> 136639 T2570 oazs.SessionTrackerImpl.run SessionTrackerImpl exited loop!
[junit4:junit4] 2> 137438 T2649 oascc.ZkStateReader$3.process WARNING ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 137438 T2649 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 137439 T2649 oascc.ZkStateReader$2.process WARNING ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 137439 T2649 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> NOTE: test params are: codec=Lucene41, sim=DefaultSimilarity, locale=da_DK, timezone=SystemV/AST4ADT
[junit4:junit4] 2> NOTE: Linux 3.2.0-38-generic i386/Oracle Corporation 1.8.0-ea (32-bit)/cpus=8,threads=2,free=36852112,total=145764352
[junit4:junit4] 2> NOTE: All tests run in this JVM: [PrimitiveFieldTypeTest, TestStressVersions, SolrCmdDistributorTest, TestArbitraryIndexDir, SuggesterFSTTest, TestSolrIndexConfig, SuggesterWFSTTest, TestCodecSupport, TestFoldingMultitermQuery, SearchHandlerTest, TestSolrDiscoveryProperties, TestFiltering, TestSurroundQueryParser, DebugComponentTest, TestRandomDVFaceting, StatelessScriptUpdateProcessorFactoryTest, SpatialFilterTest, SolrRequestParserTest, TestZkChroot, TestPHPSerializedResponseWriter, LeaderElectionIntegrationTest, TestPseudoReturnFields, TestWordDelimiterFilterFactory, SampleTest, TestLazyCores, TestValueSourceCache, RAMDirectoryFactoryTest, TestSearchPerf, SolrPluginUtilsTest, SpellCheckComponentTest, MoreLikeThisHandlerTest, SuggesterTest, BadComponentTest, TestFuzzyAnalyzedSuggestions, SimpleFacetsTest, OverseerCollectionProcessorTest, TestIndexSearcher, SpellCheckCollatorTest, FileBasedSpellCheckerTest, UpdateRequestProcessorFactoryTest, TestSolrXMLSerializer, ShardRoutingCustomTest, SynonymTokenizerTest, DistributedSpellCheckComponentTest, SOLR749Test, TestPostingsSolrHighlighter, TestFaceting, ClusterStateTest, DateMathParserTest, TestBinaryResponseWriter, TestExtendedDismaxParser, TestSolr4Spatial, SyncSliceTest, TestOmitPositions, TestUpdate, BasicDistributedZk2Test, TestLMDirichletSimilarityFactory, TestAtomicUpdateErrorCases, TestMergePolicyConfig, DistanceFunctionTest, IndexBasedSpellCheckerTest, TestBinaryField, CoreAdminHandlerTest, RecoveryZkTest, FullSolrCloudDistribCmdsTest, TestReplicationHandler, TestRealTimeGet, ZkControllerTest, TestRandomFaceting, WordBreakSolrSpellCheckerTest, TestHashPartitioner, TestJoin, SortByFunctionTest, SignatureUpdateProcessorFactoryTest, DirectSolrConnectionTest, CacheHeaderTest, PolyFieldTest, MBeansHandlerTest, IndexSchemaTest, BinaryUpdateRequestHandlerTest, MinimalSchemaTest, XmlUpdateRequestHandlerTest, PingRequestHandlerTest, TermVectorComponentTest, DirectUpdateHandlerOptimizeTest, TestReversedWildcardFilterFactory, JSONWriterTest, TestQueryUtils, UniqFieldsUpdateProcessorFactoryTest, ShowFileRequestHandlerTest, TestPropInject, NoCacheHeaderTest, DefaultValueUpdateProcessorTest, TestQuerySenderListener, NumericFieldsTest, OutputWriterTest, TestConfig, TestPhraseSuggestions, LegacyHTMLStripCharFilterTest, TestDefaultSimilarityFactory, PluginInfoTest, ResourceLoaderTest, DateFieldTest, TestBadConfig, TestUtils, PreAnalyzedFieldTest, ZkNodePropsTest, CircularListTest, AnalysisAfterCoreReloadTest, TestDocumentBuilder, PathHierarchyTokenizerFactoryTest, TestCharFilters, ClusterStateUpdateTest, UnloadDistributedZkTest]
[junit4:junit4] Completed on J0 in 137.64s, 1 test, 1 failure <<< FAILURES!
[...truncated 75 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:381: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:361: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:39: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build.xml:183: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/common-build.xml:449: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:1207: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:870: There were test failures: 261 suites, 1101 tests, 1 error, 1 failure, 13 ignored (7 assumptions)
Total time: 41 minutes 12 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Description set: Java: 32bit/jdk1.8.0-ea-b79 -client -XX:+UseSerialGC
Email was triggered for: Failure
Sending email for trigger: Failure