You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2015/05/23 23:51:18 UTC
[JENKINS] Lucene-Solr-Tests-trunk-Java8 - Build # 8 - Still Failing
Build: https://builds.apache.org/job/Lucene-Solr-Tests-trunk-Java8/8/
4 tests failed.
FAILED: org.apache.solr.cloud.TestSolrCloudWithKerberos.testKerberizedSolr
Error Message:
Error from server at http://127.0.0.1:47301/sa_xas: Expected mime type application/octet-stream but got text/html. <html> <head> <meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/> <title>Error 401 </title> </head> <body> <h2>HTTP ERROR: 401</h2> <p>Problem accessing /sa_xas/admin/collections. Reason: <pre> Authentication required</pre></p> <hr /><i><small>Powered by Jetty://</small></i> </body> </html>
Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:47301/sa_xas: Expected mime type application/octet-stream but got text/html. <html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/>
<title>Error 401 </title>
</head>
<body>
<h2>HTTP ERROR: 401</h2>
<p>Problem accessing /sa_xas/admin/collections. Reason:
<pre> Authentication required</pre></p>
<hr /><i><small>Powered by Jetty://</small></i>
</body>
</html>
at __randomizedtesting.SeedInfo.seed([7E178E8EB7F6CF8F:D50E5849FD970FB8]:0)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:529)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:235)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:227)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:376)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:328)
at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1086)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:856)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:799)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:135)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:152)
at org.apache.solr.cloud.TestSolrCloudWithKerberos.testKerberizedSolr(TestSolrCloudWithKerberos.java:157)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:960)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:935)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
at java.lang.Thread.run(Thread.java:745)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.TestSolrCloudWithKerberos
Error Message:
Some resources were not closed, shutdown, or released.
Stack Trace:
java.lang.AssertionError: Some resources were not closed, shutdown, or released.
at __randomizedtesting.SeedInfo.seed([7E178E8EB7F6CF8F]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.SolrTestCaseJ4.afterClass(SolrTestCaseJ4.java:234)
at sun.reflect.GeneratedMethodAccessor37.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:799)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
at java.lang.Thread.run(Thread.java:745)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.TestSolrCloudWithKerberos
Error Message:
5 threads leaked from SUITE scope at org.apache.solr.cloud.TestSolrCloudWithKerberos: 1) Thread[id=5806, name=zkCallback-803-thread-2, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithKerberos] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) 2) Thread[id=5802, name=TEST-TestSolrCloudWithKerberos.testKerberizedSolr-seed#[7E178E8EB7F6CF8F]-SendThread(127.0.0.1:54508), state=TIMED_WAITING, group=TGRP-TestSolrCloudWithKerberos] at java.lang.Thread.sleep(Native Method) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:994) 3) Thread[id=5805, name=zkCallback-803-thread-1, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithKerberos] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) 4) Thread[id=5841, name=zkCallback-803-thread-3, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithKerberos] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) 5) Thread[id=5803, name=TEST-TestSolrCloudWithKerberos.testKerberizedSolr-seed#[7E178E8EB7F6CF8F]-EventThread, state=WAITING, group=TGRP-TestSolrCloudWithKerberos] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:494)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 5 threads leaked from SUITE scope at org.apache.solr.cloud.TestSolrCloudWithKerberos:
1) Thread[id=5806, name=zkCallback-803-thread-2, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithKerberos]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
2) Thread[id=5802, name=TEST-TestSolrCloudWithKerberos.testKerberizedSolr-seed#[7E178E8EB7F6CF8F]-SendThread(127.0.0.1:54508), state=TIMED_WAITING, group=TGRP-TestSolrCloudWithKerberos]
at java.lang.Thread.sleep(Native Method)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:994)
3) Thread[id=5805, name=zkCallback-803-thread-1, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithKerberos]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
4) Thread[id=5841, name=zkCallback-803-thread-3, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithKerberos]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
5) Thread[id=5803, name=TEST-TestSolrCloudWithKerberos.testKerberizedSolr-seed#[7E178E8EB7F6CF8F]-EventThread, state=WAITING, group=TGRP-TestSolrCloudWithKerberos]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:494)
at __randomizedtesting.SeedInfo.seed([7E178E8EB7F6CF8F]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.TestSolrCloudWithKerberos
Error Message:
There are still zombie threads that couldn't be terminated: 1) Thread[id=5806, name=zkCallback-803-thread-2, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithKerberos] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) 2) Thread[id=5802, name=TEST-TestSolrCloudWithKerberos.testKerberizedSolr-seed#[7E178E8EB7F6CF8F]-SendThread(127.0.0.1:54508), state=TIMED_WAITING, group=TGRP-TestSolrCloudWithKerberos] at java.lang.Thread.sleep(Native Method) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:994) 3) Thread[id=5805, name=zkCallback-803-thread-1, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithKerberos] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) 4) Thread[id=5841, name=zkCallback-803-thread-3, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithKerberos] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
1) Thread[id=5806, name=zkCallback-803-thread-2, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithKerberos]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
2) Thread[id=5802, name=TEST-TestSolrCloudWithKerberos.testKerberizedSolr-seed#[7E178E8EB7F6CF8F]-SendThread(127.0.0.1:54508), state=TIMED_WAITING, group=TGRP-TestSolrCloudWithKerberos]
at java.lang.Thread.sleep(Native Method)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:994)
3) Thread[id=5805, name=zkCallback-803-thread-1, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithKerberos]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
4) Thread[id=5841, name=zkCallback-803-thread-3, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithKerberos]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
at __randomizedtesting.SeedInfo.seed([7E178E8EB7F6CF8F]:0)
Build Log:
[...truncated 9651 lines...]
[junit4] Suite: org.apache.solr.cloud.TestSolrCloudWithKerberos
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithKerberos 7E178E8EB7F6CF8F-001/init-core-data-001
[junit4] 2> 441308 T5642 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /sa_xas/
[junit4] 2> 446691 T5642 oadsc.DefaultDirectoryService.showSecurityWarnings WARN You didn't change the admin password of directory service instance 'DefaultKrbServer'. Please update the admin password as soon as possible to prevent a possible security breach.
[junit4] 2> 447310 T5642 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 2> 447311 T5651 oasc.ZkTestServer$2$1.setClientPort client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 447311 T5651 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 447411 T5642 oasc.ZkTestServer.run start zk server on port:54508
[junit4] 2> 447411 T5642 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
[junit4] 2> 447412 T5642 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 447413 T5656 oaz.ClientCnxn$SendThread.startConnect WARN SASL configuration failed: javax.security.auth.login.LoginException: No JAAS configuration section named 'Client' was found in specified JAAS configuration file: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithKerberos 7E178E8EB7F6CF8F-001/tempDir-001/minikdc/jaas-client.conf'. Will continue connection to Zookeeper server without SASL authentication, if Zookeeper server allows it.
[junit4] 2> 447414 T5658 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1f1214ba name:ZooKeeperConnection Watcher:127.0.0.1:54508 got event WatchedEvent state:AuthFailed type:None path:null path:null type:None
[junit4] 2> 447415 T5658 oascc.ConnectionManager.process WARN zkClient received AuthFailed
[junit4] 2> 447416 T5658 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1f1214ba name:ZooKeeperConnection Watcher:127.0.0.1:54508 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 447416 T5642 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 447416 T5642 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
[junit4] 2> 447417 T5642 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 447420 T5642 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
[junit4] 2> 447421 T5642 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 447421 T5659 oaz.ClientCnxn$SendThread.startConnect WARN SASL configuration failed: javax.security.auth.login.LoginException: No JAAS configuration section named 'Client' was found in specified JAAS configuration file: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithKerberos 7E178E8EB7F6CF8F-001/tempDir-001/minikdc/jaas-client.conf'. Will continue connection to Zookeeper server without SASL authentication, if Zookeeper server allows it.
[junit4] 2> 447422 T5661 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@ca6fbdd name:ZooKeeperConnection Watcher:127.0.0.1:54508/solr got event WatchedEvent state:AuthFailed type:None path:null path:null type:None
[junit4] 2> 447422 T5661 oascc.ConnectionManager.process WARN zkClient received AuthFailed
[junit4] 2> 447422 T5661 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@ca6fbdd name:ZooKeeperConnection Watcher:127.0.0.1:54508/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 447423 T5642 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 447423 T5642 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
[junit4] 2> 447424 T5642 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4] 2> 447426 T5642 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4] 2> 447428 T5642 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4] 2> 447430 T5642 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4] 2> 447431 T5642 oasc.AbstractZkTestCase.putConfig put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 447432 T5642 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4] 2> 447434 T5642 oasc.AbstractZkTestCase.putConfig put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4] 2> 447434 T5642 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4] 2> 447437 T5642 oasc.AbstractZkTestCase.putConfig put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 447437 T5642 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 447439 T5642 oasc.AbstractZkTestCase.putConfig put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 447439 T5642 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4] 2> 447441 T5642 oasc.AbstractZkTestCase.putConfig put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 447441 T5642 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4] 2> 447443 T5642 oasc.AbstractZkTestCase.putConfig put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 447443 T5642 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4] 2> 447446 T5642 oasc.AbstractZkTestCase.putConfig put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 447446 T5642 oascc.SolrZkClient.makePath makePath: /configs/conf1/enumsConfig.xml
[junit4] 2> 447448 T5642 oasc.AbstractZkTestCase.putConfig put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 447449 T5642 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4] 2> 447451 T5642 oasc.AbstractZkTestCase.putConfig put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 447451 T5642 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 447453 T5642 oasc.AbstractZkTestCase.putConfig put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 447454 T5642 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4] 2> 447456 T5642 oasc.AbstractZkTestCase.putConfig put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 447456 T5642 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4] 2> 447459 T5642 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
[junit4] 2> 447460 T5642 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 447462 T5662 oaz.ClientCnxn$SendThread.startConnect WARN SASL configuration failed: javax.security.auth.login.LoginException: No JAAS configuration section named 'Client' was found in specified JAAS configuration file: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithKerberos 7E178E8EB7F6CF8F-001/tempDir-001/minikdc/jaas-client.conf'. Will continue connection to Zookeeper server without SASL authentication, if Zookeeper server allows it.
[junit4] 2> 447463 T5664 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7a3bccb4 name:ZooKeeperConnection Watcher:127.0.0.1:54508/solr got event WatchedEvent state:AuthFailed type:None path:null path:null type:None
[junit4] 2> 447463 T5664 oascc.ConnectionManager.process WARN zkClient received AuthFailed
[junit4] 2> 447463 T5665 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7a3bccb4 name:ZooKeeperConnection Watcher:127.0.0.1:54508/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 447463 T5642 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 447464 T5642 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
[junit4] 2> 447795 T5642 oas.SolrTestCaseJ4.writeCoreProperties Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithKerberos 7E178E8EB7F6CF8F-001/control-001/cores/collection1
[junit4] 2> 447799 T5642 oejs.Server.doStart jetty-9.2.10.v20150310
[junit4] 2> 447804 T5642 oejsh.ContextHandler.doStart Started o.e.j.s.ServletContextHandler@ff755e9{/sa_xas,null,AVAILABLE}
[junit4] 2> 447805 T5642 oejs.AbstractConnector.doStart Started ServerConnector@3502207{HTTP/1.1}{127.0.0.1:57696}
[junit4] 2> 447805 T5642 oejs.Server.doStart Started @449680ms
[junit4] 2> 447805 T5642 oascse.JettySolrRunner$1.lifeCycleStarted Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithKerberos 7E178E8EB7F6CF8F-001/tempDir-002/control/data, hostContext=/sa_xas, hostPort=57696, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithKerberos 7E178E8EB7F6CF8F-001/control-001/cores}
[junit4] 2> 447806 T5642 oass.SolrDispatchFilter.init SolrDispatchFilter.init()sun.misc.Launcher$AppClassLoader@c387f44
[junit4] 2> 447806 T5642 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithKerberos 7E178E8EB7F6CF8F-001/control-001/'
[junit4] 2> 447830 T5642 oasc.SolrXmlConfig.fromFile Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithKerberos 7E178E8EB7F6CF8F-001/control-001/solr.xml
[junit4] 2> 447836 T5642 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithKerberos 7E178E8EB7F6CF8F-001/control-001/cores
[junit4] 2> 447836 T5642 oasc.CoreContainer.<init> New CoreContainer 521419505
[junit4] 2> 447836 T5642 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithKerberos 7E178E8EB7F6CF8F-001/control-001/]
[junit4] 2> 447836 T5642 oasc.CoreContainer.load loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithKerberos 7E178E8EB7F6CF8F-001/control-001/lib
[junit4] 2> 447837 T5642 oasc.SolrResourceLoader.addToClassLoader WARN Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithKerberos 7E178E8EB7F6CF8F-001/control-001/lib).
[junit4] 2> 447844 T5642 oashc.HttpShardHandlerFactory.init created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
[junit4] 2> 447846 T5642 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 447846 T5642 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 447847 T5642 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 447847 T5642 oasc.CoreContainer.load Node Name: 127.0.0.1
[junit4] 2> 447847 T5642 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:54508/solr
[junit4] 2> 447847 T5642 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 447847 T5642 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
[junit4] 2> 447848 T5642 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 447855 T5677 oaz.ClientCnxn$SendThread.startConnect WARN SASL configuration failed: javax.security.auth.login.LoginException: No JAAS configuration section named 'Client' was found in specified JAAS configuration file: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithKerberos 7E178E8EB7F6CF8F-001/tempDir-001/minikdc/jaas-client.conf'. Will continue connection to Zookeeper server without SASL authentication, if Zookeeper server allows it.
[junit4] 2> 447856 T5679 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@70fbafd name:ZooKeeperConnection Watcher:127.0.0.1:54508 got event WatchedEvent state:AuthFailed type:None path:null path:null type:None
[junit4] 2> 447856 T5679 oascc.ConnectionManager.process WARN zkClient received AuthFailed
[junit4] 2> 447858 T5679 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@70fbafd name:ZooKeeperConnection Watcher:127.0.0.1:54508 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 447858 T5642 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 447859 T5642 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
[junit4] 2> 447864 T5642 n:127.0.0.1:57696_sa_xas oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 447873 T5680 n:127.0.0.1:57696_sa_xas oaz.ClientCnxn$SendThread.startConnect WARN SASL configuration failed: javax.security.auth.login.LoginException: No JAAS configuration section named 'Client' was found in specified JAAS configuration file: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithKerberos 7E178E8EB7F6CF8F-001/tempDir-001/minikdc/jaas-client.conf'. Will continue connection to Zookeeper server without SASL authentication, if Zookeeper server allows it.
[junit4] 2> 447888 T5682 n:127.0.0.1:57696_sa_xas oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@54a1612 name:ZooKeeperConnection Watcher:127.0.0.1:54508/solr got event WatchedEvent state:AuthFailed type:None path:null path:null type:None
[junit4] 2> 447888 T5682 n:127.0.0.1:57696_sa_xas oascc.ConnectionManager.process WARN zkClient received AuthFailed
[junit4] 2> 447888 T5683 n:127.0.0.1:57696_sa_xas oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@54a1612 name:ZooKeeperConnection Watcher:127.0.0.1:54508/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 447888 T5642 n:127.0.0.1:57696_sa_xas oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 447890 T5642 n:127.0.0.1:57696_sa_xas oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 447892 T5642 n:127.0.0.1:57696_sa_xas oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 447894 T5642 n:127.0.0.1:57696_sa_xas oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
[junit4] 2> 447896 T5642 n:127.0.0.1:57696_sa_xas oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
[junit4] 2> 447897 T5642 n:127.0.0.1:57696_sa_xas oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
[junit4] 2> 447899 T5642 n:127.0.0.1:57696_sa_xas oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 447900 T5642 n:127.0.0.1:57696_sa_xas oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 447901 T5642 n:127.0.0.1:57696_sa_xas oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 447902 T5642 n:127.0.0.1:57696_sa_xas oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:57696_sa_xas
[junit4] 2> 447903 T5642 n:127.0.0.1:57696_sa_xas oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:57696_sa_xas
[junit4] 2> 447904 T5642 n:127.0.0.1:57696_sa_xas oascc.SolrZkClient.makePath makePath: /overseer_elect
[junit4] 2> 447905 T5642 n:127.0.0.1:57696_sa_xas oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 447906 T5642 n:127.0.0.1:57696_sa_xas oasc.Overseer.close Overseer (id=null) closing
[junit4] 2> 447907 T5642 n:127.0.0.1:57696_sa_xas oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:57696_sa_xas
[junit4] 2> 447908 T5642 n:127.0.0.1:57696_sa_xas oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 447909 T5642 n:127.0.0.1:57696_sa_xas oasc.Overseer.start Overseer (id=93874899301171204-127.0.0.1:57696_sa_xas-n_0000000000) starting
[junit4] 2> 447910 T5642 n:127.0.0.1:57696_sa_xas oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 447916 T5642 n:127.0.0.1:57696_sa_xas oasc.OverseerAutoReplicaFailoverThread.<init> Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
[junit4] 2> 447953 T5684 n:127.0.0.1:57696_sa_xas oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 447961 T5685 n:127.0.0.1:57696_sa_xas oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 447974 T5642 n:127.0.0.1:57696_sa_xas oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 447977 T5642 n:127.0.0.1:57696_sa_xas oasc.CoreContainer.initializeAuthenticationPlugin Authentication plugin class obtained from ZK: org.apache.solr.security.KerberosPlugin
[junit4] 2> 447984 T5642 n:127.0.0.1:57696_sa_xas oass.KerberosPlugin.init Params: {token.valid=30, kerberos.principal=HTTP/127.0.0.1, cookie.domain=127.0.0.1, kerberos.keytab=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithKerberos 7E178E8EB7F6CF8F-001/tempDir-001/minikdc/keytabs, cookie.path=/, type=kerberos, kerberos.name.rules=DEFAULT}
[junit4] 2> 448033 T5642 n:127.0.0.1:57696_sa_xas oasc.CoreContainer.initializeAuthenticationPlugin Reconfiguring the shard handler factory and update shard handler.
[junit4] 2> 448034 T5642 n:127.0.0.1:57696_sa_xas oashc.HttpShardHandlerFactory.reconfigureHttpClient Reconfiguring the default client with: org.apache.solr.client.solrj.impl.Krb5HttpClientConfigurer@1a634ce2
[junit4] 2> 448034 T5642 n:127.0.0.1:57696_sa_xas oascsi.Krb5HttpClientConfigurer.configure Setting up SPNego auth with config: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithKerberos 7E178E8EB7F6CF8F-001/tempDir-001/minikdc/jaas-client.conf
[junit4] 2> 448037 T5642 n:127.0.0.1:57696_sa_xas oasu.UpdateShardHandler.reconfigureHttpClient Reconfiguring the default client with: org.apache.solr.client.solrj.impl.Krb5HttpClientConfigurer@1a634ce2
[junit4] 2> 448038 T5642 n:127.0.0.1:57696_sa_xas oascsi.Krb5HttpClientConfigurer.configure Setting up SPNego auth with config: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithKerberos 7E178E8EB7F6CF8F-001/tempDir-001/minikdc/jaas-client.conf
[junit4] 2> 448040 T5642 n:127.0.0.1:57696_sa_xas oasc.CorePropertiesLocator.discover Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithKerberos 7E178E8EB7F6CF8F-001/control-001/cores
[junit4] 2> 448041 T5642 n:127.0.0.1:57696_sa_xas oasc.CoreDescriptor.<init> CORE DESCRIPTOR: {name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, loadOnStartup=true, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithKerberos 7E178E8EB7F6CF8F-001/control-001/cores/collection1, collection=control_collection, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithKerberos 7E178E8EB7F6CF8F-001/control-001/cores/collection1/, coreNodeName=, dataDir=data/, shard=}
[junit4] 2> 448041 T5642 n:127.0.0.1:57696_sa_xas oasc.CorePropertiesLocator.discoverUnder Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithKerberos 7E178E8EB7F6CF8F-001/control-001/cores/collection1/
[junit4] 2> 448041 T5642 n:127.0.0.1:57696_sa_xas oasc.CorePropertiesLocator.discover Found 1 core definitions
[junit4] 2> 448042 T5688 n:127.0.0.1:57696_sa_xas c:control_collection x:collection1 oasc.ZkController.publish publishing core=collection1 state=down collection=control_collection
[junit4] 2> 448042 T5688 n:127.0.0.1:57696_sa_xas c:control_collection x:collection1 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 448043 T5688 n:127.0.0.1:57696_sa_xas oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 448045 T5683 n:127.0.0.1:57696_sa_xas oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
[junit4] 2> 448046 T5684 n:127.0.0.1:57696_sa_xas oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"http://127.0.0.1:57696/sa_xas",
[junit4] 2> "node_name":"127.0.0.1:57696_sa_xas",
[junit4] 2> "numShards":"1",
[junit4] 2> "state":"down",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"control_collection",
[junit4] 2> "operation":"state"} current state version: 0
[junit4] 2> 448047 T5684 n:127.0.0.1:57696_sa_xas oasco.ReplicaMutator.updateState Update state numShards=1 message={
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"http://127.0.0.1:57696/sa_xas",
[junit4] 2> "node_name":"127.0.0.1:57696_sa_xas",
[junit4] 2> "numShards":"1",
[junit4] 2> "state":"down",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"control_collection",
[junit4] 2> "operation":"state"}
[junit4] 2> 448047 T5684 n:127.0.0.1:57696_sa_xas oasco.ClusterStateMutator.createCollection building a new cName: control_collection
[junit4] 2> 448047 T5684 n:127.0.0.1:57696_sa_xas oasco.ReplicaMutator.updateState Assigning new node to shard shard=shard1
[junit4] 2> 448049 T5683 n:127.0.0.1:57696_sa_xas oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 449043 T5688 n:127.0.0.1:57696_sa_xas oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 449044 T5688 n:127.0.0.1:57696_sa_xas oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4] 2> 449044 T5688 n:127.0.0.1:57696_sa_xas oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 449044 T5688 n:127.0.0.1:57696_sa_xas oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
[junit4] 2> 449045 T5688 n:127.0.0.1:57696_sa_xas oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 449045 T5688 n:127.0.0.1:57696_sa_xas oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithKerberos 7E178E8EB7F6CF8F-001/control-001/cores/collection1/'
[junit4] 2> 449067 T5688 n:127.0.0.1:57696_sa_xas oasc.Config.<init> loaded config solrconfig.xml with version 0
[junit4] 2> 449073 T5688 n:127.0.0.1:57696_sa_xas oasc.SolrConfig.refreshRequestParams current version of requestparams : -1
[junit4] 2> 449101 T5688 n:127.0.0.1:57696_sa_xas oasc.SolrConfig.<init> Using Lucene MatchVersion: 6.0.0
[junit4] 2> 449109 T5688 n:127.0.0.1:57696_sa_xas oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 449110 T5688 n:127.0.0.1:57696_sa_xas oass.IndexSchema.readSchema Reading Solr Schema from /configs/conf1/schema.xml
[junit4] 2> 449115 T5688 n:127.0.0.1:57696_sa_xas oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 449235 T5688 n:127.0.0.1:57696_sa_xas oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 449240 T5688 n:127.0.0.1:57696_sa_xas oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 449241 T5688 n:127.0.0.1:57696_sa_xas oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 449249 T5688 n:127.0.0.1:57696_sa_xas oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 449252 T5688 n:127.0.0.1:57696_sa_xas oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 449253 T5688 n:127.0.0.1:57696_sa_xas oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 449254 T5688 n:127.0.0.1:57696_sa_xas oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 449254 T5688 n:127.0.0.1:57696_sa_xas oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 449254 T5688 n:127.0.0.1:57696_sa_xas oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 449255 T5688 n:127.0.0.1:57696_sa_xas oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 449255 T5688 n:127.0.0.1:57696_sa_xas oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 449255 T5688 n:127.0.0.1:57696_sa_xas oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection control_collection
[junit4] 2> 449256 T5688 n:127.0.0.1:57696_sa_xas x:collection1 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 449256 T5688 n:127.0.0.1:57696_sa_xas x:collection1 oasc.SolrCore.<init> [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithKerberos 7E178E8EB7F6CF8F-001/control-001/cores/collection1/], dataDir=[null]
[junit4] 2> 449256 T5688 n:127.0.0.1:57696_sa_xas x:collection1 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@2817f3b1
[junit4] 2> 449257 T5688 n:127.0.0.1:57696_sa_xas x:collection1 oasc.CachingDirectoryFactory.get return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithKerberos 7E178E8EB7F6CF8F-001/control-001/cores/collection1/data
[junit4] 2> 449257 T5688 n:127.0.0.1:57696_sa_xas x:collection1 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithKerberos 7E178E8EB7F6CF8F-001/control-001/cores/collection1/data/index/
[junit4] 2> 449257 T5688 n:127.0.0.1:57696_sa_xas x:collection1 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithKerberos 7E178E8EB7F6CF8F-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
[junit4] 2> 449257 T5688 n:127.0.0.1:57696_sa_xas x:collection1 oasc.CachingDirectoryFactory.get return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithKerberos 7E178E8EB7F6CF8F-001/control-001/cores/collection1/data/index
[junit4] 2> 449258 T5688 n:127.0.0.1:57696_sa_xas x:collection1 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=30, maxMergeAtOnceExplicit=12, maxMergedSegmentMB=72.904296875, floorSegmentMB=1.474609375, forceMergeDeletesPctAllowed=11.71015669263026, segmentsPerTier=50.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.551230275170388
[junit4] 2> 449258 T5688 n:127.0.0.1:57696_sa_xas x:collection1 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@166e2cd2 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@6f314c87),segFN=segments_1,generation=1}
[junit4] 2> 449259 T5688 n:127.0.0.1:57696_sa_xas x:collection1 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 449262 T5688 n:127.0.0.1:57696_sa_xas x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 449262 T5688 n:127.0.0.1:57696_sa_xas x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 449262 T5688 n:127.0.0.1:57696_sa_xas x:collection1 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 449262 T5688 n:127.0.0.1:57696_sa_xas x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 449263 T5688 n:127.0.0.1:57696_sa_xas x:collection1 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 449263 T5688 n:127.0.0.1:57696_sa_xas x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 449263 T5688 n:127.0.0.1:57696_sa_xas x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 449263 T5688 n:127.0.0.1:57696_sa_xas x:collection1 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 449263 T5688 n:127.0.0.1:57696_sa_xas x:collection1 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 449265 T5688 n:127.0.0.1:57696_sa_xas x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 449266 T5688 n:127.0.0.1:57696_sa_xas x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 449267 T5688 n:127.0.0.1:57696_sa_xas x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 449268 T5688 n:127.0.0.1:57696_sa_xas x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 449271 T5688 n:127.0.0.1:57696_sa_xas x:collection1 oasc.RequestHandlers.initHandlersFromConfig Registered paths: /admin/mbeans,standard,/update/csv,/update/json/docs,/admin/luke,/admin/segments,/get,/admin/system,/replication,/admin/properties,/config,/schema,/admin/plugins,/admin/logging,/update/json,/admin/threads,/admin/ping,/update,/admin/file
[junit4] 2> 449272 T5688 n:127.0.0.1:57696_sa_xas x:collection1 oasc.SolrCore.initStatsCache Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
[junit4] 2> 449272 T5688 n:127.0.0.1:57696_sa_xas x:collection1 oasu.UpdateHandler.<init> Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 449272 T5688 n:127.0.0.1:57696_sa_xas x:collection1 oasu.UpdateLog.init Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 449274 T5688 n:127.0.0.1:57696_sa_xas x:collection1 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 449275 T5688 n:127.0.0.1:57696_sa_xas x:collection1 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 449275 T5688 n:127.0.0.1:57696_sa_xas x:collection1 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=37, maxMergeAtOnceExplicit=13, maxMergedSegmentMB=92.451171875, floorSegmentMB=1.6708984375, forceMergeDeletesPctAllowed=23.40399930118291, segmentsPerTier=17.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.6173965239428705
[junit4] 2> 449276 T5688 n:127.0.0.1:57696_sa_xas x:collection1 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@166e2cd2 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@6f314c87),segFN=segments_1,generation=1}
[junit4] 2> 449276 T5688 n:127.0.0.1:57696_sa_xas x:collection1 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 449281 T5688 n:127.0.0.1:57696_sa_xas x:collection1 oass.SolrIndexSearcher.<init> Opening Searcher@2431881e[collection1] main
[junit4] 2> 449282 T5688 n:127.0.0.1:57696_sa_xas x:collection1 oasu.UpdateLog.onFirstSearcher On first searcher opened, looking up max value of version field
[junit4] 2> 449282 T5688 n:127.0.0.1:57696_sa_xas x:collection1 oasu.VersionInfo.getMaxVersionFromIndex Refreshing highest value of _version_ for 65536 version buckets from index
[junit4] 2> 449282 T5688 n:127.0.0.1:57696_sa_xas x:collection1 oasu.VersionInfo.getMaxVersionFromIndex WARN No terms found for _version_, cannot seed version bucket highest value from index
[junit4] 2> 449282 T5688 n:127.0.0.1:57696_sa_xas x:collection1 oasu.UpdateLog.seedBucketsWithHighestVersion WARN Could not find max version in index or recent updates, using new clock 1501998390884433920
[junit4] 2> 449286 T5688 n:127.0.0.1:57696_sa_xas x:collection1 oasu.UpdateLog.seedBucketsWithHighestVersion Took 4 ms to seed version buckets with highest version 1501998390884433920
[junit4] 2> 449286 T5688 n:127.0.0.1:57696_sa_xas x:collection1 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
[junit4] 2> 449287 T5688 n:127.0.0.1:57696_sa_xas x:collection1 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 449287 T5688 n:127.0.0.1:57696_sa_xas x:collection1 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
[junit4] 2> 449288 T5688 n:127.0.0.1:57696_sa_xas x:collection1 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 449288 T5688 n:127.0.0.1:57696_sa_xas x:collection1 oasr.RestManager.init Initializing RestManager with initArgs: {}
[junit4] 2> 449288 T5688 n:127.0.0.1:57696_sa_xas x:collection1 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 449289 T5688 n:127.0.0.1:57696_sa_xas x:collection1 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json
[junit4] 2> 449289 T5688 n:127.0.0.1:57696_sa_xas x:collection1 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 449289 T5688 n:127.0.0.1:57696_sa_xas x:collection1 oasr.RestManager.init Initializing 0 registered ManagedResources
[junit4] 2> 449289 T5688 n:127.0.0.1:57696_sa_xas x:collection1 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4] 2> 449290 T5688 n:127.0.0.1:57696_sa_xas x:collection1 oasc.ZkController.getConfDirListeners watch zkdir /configs/conf1
[junit4] 2> 449290 T5689 n:127.0.0.1:57696_sa_xas x:collection1 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2431881e[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 449290 T5688 n:127.0.0.1:57696_sa_xas x:collection1 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 449292 T5692 n:127.0.0.1:57696_sa_xas c:control_collection s:shard1 x:collection1 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:57696/sa_xas collection:control_collection shard:shard1
[junit4] 2> 449292 T5642 n:127.0.0.1:57696_sa_xas oascsi.Krb5HttpClientConfigurer.configure Setting up SPNego auth with config: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithKerberos 7E178E8EB7F6CF8F-001/tempDir-001/minikdc/jaas-client.conf
[junit4] 2> 449292 T5642 n:127.0.0.1:57696_sa_xas oass.SolrDispatchFilter.init user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1
[junit4] 2> 449292 T5642 n:127.0.0.1:57696_sa_xas oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 449292 T5692 n:127.0.0.1:57696_sa_xas c:control_collection s:shard1 x:collection1 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4] 2> 449294 T5642 oascsi.Krb5HttpClientConfigurer.configure Setting up SPNego auth with config: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithKerberos 7E178E8EB7F6CF8F-001/tempDir-001/minikdc/jaas-client.conf
[junit4] 2> 449294 T5642 oascsi.Krb5HttpClientConfigurer.configure Setting up SPNego auth with config: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithKerberos 7E178E8EB7F6CF8F-001/tempDir-001/minikdc/jaas-client.conf
[junit4] 2> 449295 T5642 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
[junit4] 2> 449296 T5642 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 449302 T5692 n:127.0.0.1:57696_sa_xas c:control_collection s:shard1 x:collection1 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 449302 T5693 oaz.ClientCnxn$SendThread.startConnect WARN SASL configuration failed: javax.security.auth.login.LoginException: No JAAS configuration section named 'Client' was found in specified JAAS configuration file: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithKerberos 7E178E8EB7F6CF8F-001/tempDir-001/minikdc/jaas-client.conf'. Will continue connection to Zookeeper server without SASL authentication, if Zookeeper server allows it.
[junit4] 2> 449303 T5683 n:127.0.0.1:57696_sa_xas oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
[junit4] 2> 449304 T5684 n:127.0.0.1:57696_sa_xas oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"leader",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"control_collection"} current state version: 1
[junit4] 2> 449309 T5692 n:127.0.0.1:57696_sa_xas c:control_collection s:shard1 x:collection1 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 449310 T5692 n:127.0.0.1:57696_sa_xas c:control_collection s:shard1 x:collection1 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> 449310 T5695 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@51d3fd52 name:ZooKeeperConnection Watcher:127.0.0.1:54508/solr got event WatchedEvent state:AuthFailed type:None path:null path:null type:None
[junit4] 2> 449310 T5695 oascc.ConnectionManager.process WARN zkClient received AuthFailed
[junit4] 2> ASYNC NEW_CORE C565 name=collection1 org.apache.solr.core.SolrCore@2581dd64 url=http://127.0.0.1:57696/sa_xas/collection1 node=127.0.0.1:57696_sa_xas C565_STATE=coll:control_collection core:collection1 props:{core=collection1, base_url=http://127.0.0.1:57696/sa_xas, node_name=127.0.0.1:57696_sa_xas, state=down}
[junit4] 2> 449310 T5692 n:127.0.0.1:57696_sa_xas c:control_collection s:shard1 x:collection1 C565 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:57696/sa_xas/collection1/
[junit4] 2> 449310 T5692 n:127.0.0.1:57696_sa_xas c:control_collection s:shard1 x:collection1 C565 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 449310 T5692 n:127.0.0.1:57696_sa_xas c:control_collection s:shard1 x:collection1 C565 oasc.SyncStrategy.syncToMe http://127.0.0.1:57696/sa_xas/collection1/ has no replicas
[junit4] 2> 449310 T5692 n:127.0.0.1:57696_sa_xas c:control_collection s:shard1 x:collection1 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:57696/sa_xas/collection1/ shard1
[junit4] 2> 449311 T5692 n:127.0.0.1:57696_sa_xas c:control_collection s:shard1 x:collection1 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4] 2> 449312 T5696 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@51d3fd52 name:ZooKeeperConnection Watcher:127.0.0.1:54508/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 449313 T5642 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 449314 T5642 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
[junit4] 2> 449314 T5642 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 449315 T5683 n:127.0.0.1:57696_sa_xas oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
[junit4] 2> 449316 T5684 n:127.0.0.1:57696_sa_xas oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"leader",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"control_collection",
[junit4] 2> "base_url":"http://127.0.0.1:57696/sa_xas",
[junit4] 2> "core":"collection1",
[junit4] 2> "state":"active"} current state version: 1
[junit4] 2> 449318 T5642 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 449318 T5642 oasc.AbstractFullDistribZkTestBase.createJettys Creating collection1 with stateFormat=2
[junit4] 2> 449318 T5642 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
[junit4] 2> 449319 T5642 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 449322 T5697 oaz.ClientCnxn$SendThread.startConnect WARN SASL configuration failed: javax.security.auth.login.LoginException: No JAAS configuration section named 'Client' was found in specified JAAS configuration file: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithKerberos 7E178E8EB7F6CF8F-001/tempDir-001/minikdc/jaas-client.conf'. Will continue connection to Zookeeper server without SASL authentication, if Zookeeper server allows it.
[junit4] 2> 449323 T5699 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@21e9b341 name:ZooKeeperConnection Watcher:127.0.0.1:54508/solr got event WatchedEvent state:AuthFailed type:None path:null path:null type:None
[junit4] 2> 449323 T5699 oascc.ConnectionManager.process WARN zkClient received AuthFailed
[junit4] 2> 449323 T5699 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@21e9b341 name:ZooKeeperConnection Watcher:127.0.0.1:54508/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 449323 T5642 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 449324 T5642 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
[junit4] 2> 449325 T5683 n:127.0.0.1:57696_sa_xas oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
[junit4] 2> 449330 T5684 n:127.0.0.1:57696_sa_xas oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"create",
[junit4] 2> "name":"collection1",
[junit4] 2> "numShards":"2",
[junit4] 2> "stateFormat":"2"} current state version: 1
[junit4] 2> 449331 T5684 n:127.0.0.1:57696_sa_xas oasco.ClusterStateMutator.createCollection building a new cName: collection1
[junit4] 2> 449331 T5683 n:127.0.0.1:57696_sa_xas oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 449332 T5696 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 449367 T5692 n:127.0.0.1:57696_sa_xas c:control_collection s:shard1 x:collection1 oasc.ZkController.register We are http://127.0.0.1:57696/sa_xas/collection1/ and leader is http://127.0.0.1:57696/sa_xas/collection1/
[junit4] 2> 449367 T5692 n:127.0.0.1:57696_sa_xas c:control_collection s:shard1 x:collection1 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:57696/sa_xas
[junit4] 2> 449367 T5692 n:127.0.0.1:57696_sa_xas c:control_collection s:shard1 x:collection1 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 449368 T5692 n:127.0.0.1:57696_sa_xas c:control_collection s:shard1 x:collection1 oasc.ZkController.publish publishing core=collection1 state=active collection=control_collection
[junit4] 2> 449368 T5692 n:127.0.0.1:57696_sa_xas c:control_collection s:shard1 x:collection1 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 449369 T5683 n:127.0.0.1:57696_sa_xas oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
[junit4] 2> 449369 T5684 n:127.0.0.1:57696_sa_xas oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
[junit4] 2> "core":"collection1",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"http://127.0.0.1:57696/sa_xas",
[junit4] 2> "node_name":"127.0.0.1:57696_sa_xas",
[junit4] 2> "numShards":"2",
[junit4] 2> "state":"active",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"control_collection",
[junit4] 2> "operation":"state"} current state version: 2
[junit4] 2> 449370 T5684 n:127.0.0.1:57696_sa_xas oasco.ReplicaMutator.updateState Update state numShards=2 message={
[junit4] 2> "core":"collection1",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"http://127.0.0.1:57696/sa_xas",
[junit4] 2> "node_name":"127.0.0.1:57696_sa_xas",
[junit4] 2> "numShards":"2",
[junit4] 2> "state":"active",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"control_collection",
[junit4] 2> "operation":"state"}
[junit4] 2> 449371 T5684 n:127.0.0.1:57696_sa_xas oasco.ZkStateWriter.writePendingUpdates going to create_collection /collections/collection1/state.json
[junit4] 2> 449371 T5696 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 449372 T5683 n:127.0.0.1:57696_sa_xas oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 449482 T5696 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 449482 T5683 n:127.0.0.1:57696_sa_xas oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 449643 T5642 oas.SolrTestCaseJ4.writeCoreProperties Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithKerberos 7E178E8EB7F6CF8F-001/shard-1-001/cores/collection1
[junit4] 2> 449644 T5642 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithKerberos 7E178E8EB7F6CF8F-001/shard-1-001
[junit4] 2> 449645 T5642 oejs.Server.doStart jetty-9.2.10.v20150310
[junit4] 2> 449647 T5642 oejsh.ContextHandler.doStart Started o.e.j.s.ServletContextHandler@791de837{/sa_xas,null,AVAILABLE}
[junit4] 2> 449647 T5642 oejs.AbstractConnector.doStart Started ServerConnector@170bd73{HTTP/1.1}{127.0.0.1:57923}
[junit4] 2> 449647 T5642 oejs.Server.doStart Started @451523ms
[junit4] 2> 449648 T5642 oascse.JettySolrRunner$1.lifeCycleStarted Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithKerberos 7E178E8EB7F6CF8F-001/tempDir-002/jetty1, solrconfig=solrconfig.xml, hostContext=/sa_xas, hostPort=57923, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithKerberos 7E178E8EB7F6CF8F-001/shard-1-001/cores}
[junit4] 2> 449648 T5642 oascsi.Krb5HttpClientConfigurer.configure Setting up SPNego auth with config: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithKerberos 7E178E8EB7F6CF8F-001/tempDir-001/minikdc/jaas-client.conf
[junit4] 2> 449649 T5642 oass.SolrDispatchFilter.init SolrDispatchFilter.init()sun.misc.Launcher$AppClassLoader@c387f44
[junit4] 2> 449649 T5642 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithKerberos 7E178E8EB7F6CF8F-001/shard-1-001/'
[junit4] 2> 449662 T5642 oasc.SolrXmlConfig.fromFile Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithKerberos 7E178E8EB7F6CF8F-001/shard-1-001/solr.xml
[junit4] 2> 449677 T5642 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithKerberos 7E178E8EB7F6CF8F-001/shard-1-001/cores
[junit4] 2> 449677 T5642 oasc.CoreContainer.<init> New CoreContainer 345418655
[junit4] 2> 449678 T5642 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithKerberos 7E178E8EB7F6CF8F-001/shard-1-001/]
[junit4] 2> 449679 T5642 oasc.CoreContainer.load loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithKerberos 7E178E8EB7F6CF8F-001/shard-1-001/lib
[junit4] 2> 449679 T5642 oasc.SolrResourceLoader.addToClassLoader WARN Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithKerberos 7E178E8EB7F6CF8F-001/shard-1-001/lib).
[junit4] 2> 449690 T5642 oashc.HttpShardHandlerFactory.init created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
[junit4] 2> 449693 T5642 oascsi.Krb5HttpClientConfigurer.configure Setting up SPNego auth with config: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithKerberos 7E178E8EB7F6CF8F-001/tempDir-001/minikdc/jaas-client.conf
[junit4] 2> 449693 T5642 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 449694 T5642 oascsi.Krb5HttpClientConfigurer.configure Setting up SPNego auth with config: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithKerberos 7E178E8EB7F6CF8F-001/tempDir-001/minikdc/jaas-client.conf
[junit4] 2> 449695 T5642 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 449695 T5642 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 449695 T5642 oasc.CoreContainer.load Node Name: 127.0.0.1
[junit4] 2> 449695 T5642 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:54508/solr
[junit4] 2> 449696 T5642 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 449696 T5642 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
[junit4] 2> 449697 T5642 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 449698 T5711 oaz.ClientCnxn$SendThread.startConnect WARN SASL configuration failed: javax.security.auth.login.LoginException: No JAAS configuration section named 'Client' was found in specified JAAS configuration file: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithKerberos 7E178E8EB7F6CF8F-001/tempDir-001/minikdc/jaas-client.conf'. Will continue connection to Zookeeper server without SASL authentication, if Zookeeper server allows it.
[junit4] 2> 449699 T5713 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1cc072f6 name:ZooKeeperConnection Watcher:127.0.0.1:54508 got event WatchedEvent state:AuthFailed type:None path:null path:null type:None
[junit4] 2> 449700 T5713 oascc.ConnectionManager.process WARN zkClient received AuthFailed
[junit4] 2> 449701 T5713 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1cc072f6 name:ZooKeeperConnection Watcher:127.0.0.1:54508 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 449701 T5642 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 449702 T5642 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
[junit4] 2> 449706 T5642 n:127.0.0.1:57923_sa_xas oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 449710 T5714 n:127.0.0.1:57923_sa_xas oaz.ClientCnxn$SendThread.startConnect WARN SASL configuration failed: javax.security.auth.login.LoginException: No JAAS configuration section named 'Client' was found in specified JAAS configuration file: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithKerberos 7E178E8EB7F6CF8F-001/tempDir-001/minikdc/jaas-client.conf'. Will continue connection to Zookeeper server without SASL authentication, if Zookeeper server allows it.
[junit4] 2> 449712 T5716 n:127.0.0.1:57923_sa_xas oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2ad38e5e name:ZooKeeperConnection Watcher:127.0.0.1:54508/solr got event WatchedEvent state:AuthFailed type:None path:null path:null type:None
[junit4] 2> 449712 T5716 n:127.0.0.1:57923_sa_xas oascc.ConnectionManager.process WARN zkClient received AuthFailed
[junit4] 2> 449718 T5717 n:127.0.0.1:57923_sa_xas oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2ad38e5e name:ZooKeeperConnection Watcher:127.0.0.1:54508/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 449718 T5642 n:127.0.0.1:57923_sa_xas oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 449728 T5642 n:127.0.0.1:57923_sa_xas oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 450733 T5642 n:127.0.0.1:57923_sa_xas oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:57923_sa_xas
[junit4] 2> 450734 T5642 n:127.0.0.1:57923_sa_xas oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:57923_sa_xas
[junit4] 2> 450737 T5642 n:127.0.0.1:57923_sa_xas oasc.Overseer.close Overseer (id=null) closing
[junit4] 2> 450738 T5642 n:127.0.0.1:57923_sa_xas oasc.CoreContainer.initializeAuthenticationPlugin Authentication plugin class obtained from ZK: org.apache.solr.security.KerberosPlugin
[junit4] 2> 450739 T5642 n:127.0.0.1:57923_sa_xas oass.KerberosPlugin.init Params: {token.valid=30, kerberos.principal=HTTP/127.0.0.1, cookie.domain=127.0.0.1, kerberos.keytab=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithKerberos 7E178E8EB7F6CF8F-001/tempDir-001/minikdc/keytabs, cookie.path=/, type=kerberos, kerberos.name.rules=DEFAULT}
[junit4] 2> 450744 T5642 n:127.0.0.1:57923_sa_xas oasc.CoreContainer.initializeAuthenticationPlugin Reconfiguring the shard handler factory and update shard handler.
[junit4] 2> 450744 T5642 n:127.0.0.1:57923_sa_xas oashc.HttpShardHandlerFactory.reconfigureHttpClient Reconfiguring the default client with: org.apache.solr.client.solrj.impl.Krb5HttpClientConfigurer@3ff25e5d
[junit4] 2> 450744 T5642 n:127.0.0.1:57923_sa_xas oascsi.Krb5HttpClientConfigurer.configure Setting up SPNego auth with config: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithKerberos 7E178E8EB7F6CF8F-001/tempDir-001/minikdc/jaas-client.conf
[junit4] 2> 450745 T5642 n:127.0.0.1:57923_sa_xas oasu.UpdateShardHandler.reconfigureHttpClient Reconfiguring the default client with: org.apache.solr.client.solrj.impl.Krb5HttpClientConfigurer@3ff25e5d
[junit4] 2> 450745 T5642 n:127.0.0.1:57923_sa_xas oascsi.Krb5HttpClientConfigurer.configure Setting up SPNego auth with config: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithKerberos 7E178E8EB7F6CF8F-001/tempDir-001/minikdc/jaas-client.conf
[junit4] 2> 450746 T5642 n:127.0.0.1:57923_sa_xas oasc.CorePropertiesLocator.discover Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithKerberos 7E178E8EB7F6CF8F-001/shard-1-001/cores
[junit4] 2> 450747 T5642 n:127.0.0.1:57923_sa_xas oasc.CoreDescriptor.<init> CORE DESCRIPTOR: {name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, loadOnStartup=true, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithKerberos 7E178E8EB7F6CF8F-001/shard-1-001/cores/collection1, collection=collection1, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithKerberos 7E178E8EB7F6CF8F-001/shard-1-001/cores/collection1/, coreNodeName=, dataDir=data/, shard=}
[junit4] 2> 450748 T5642 n:127.0.0.1:57923_sa_xas oasc.CorePropertiesLocator.discoverUnder Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithKerberos 7E178E8EB7F6CF8F-001/shard-1-001/cores/collection1/
[junit4] 2> 450748 T5642 n:127.0.0.1:57923_sa_xas oasc.CorePropertiesLocator.discover Found 1 core definitions
[junit4] 2> 450750 T5719 n:127.0.0.1:57923_sa_xas c:collection1 x:collection1 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
[junit4] 2> 450750 T5719 n:127.0.0.1:57923_sa_xas c:collection1 x:collection1 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 450752 T5683 n:127.0.0.1:57696_sa_xas oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
[junit4] 2> 450752 T5684 n:127.0.0.1:57696_sa_xas oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"http://127.0.0.1:57923/sa_xas",
[junit4] 2> "node_name":"127.0.0.1:57923_sa_xas",
[junit4] 2> "numShards":"2",
[junit4] 2> "state":"down",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "operation":"state"} current state version: 4
[junit4] 2> 450753 T5684 n:127.0.0.1:57696_sa_xas oasco.ReplicaMutator.updateState Update state numShards=2 message={
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"http://127.0.0.1:57923/sa_xas",
[junit4] 2> "node_name":"127.0.0.1:57923_sa_xas",
[junit4] 2> "numShards":"2",
[junit4] 2> "state":"down",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"collection1",
[junit4]
[...truncated too long message...]
breville -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] ERROR 0.00s J1 | TestSolrCloudWithKerberos (suite) <<<
[junit4] > Throwable #1: java.lang.AssertionError: Some resources were not closed, shutdown, or released.
[junit4] > at __randomizedtesting.SeedInfo.seed([7E178E8EB7F6CF8F]:0)
[junit4] > at org.apache.solr.SolrTestCaseJ4.afterClass(SolrTestCaseJ4.java:234)
[junit4] > at java.lang.Thread.run(Thread.java:745)Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: 5 threads leaked from SUITE scope at org.apache.solr.cloud.TestSolrCloudWithKerberos:
[junit4] > 1) Thread[id=5806, name=zkCallback-803-thread-2, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithKerberos]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
[junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
[junit4] > at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] > 2) Thread[id=5802, name=TEST-TestSolrCloudWithKerberos.testKerberizedSolr-seed#[7E178E8EB7F6CF8F]-SendThread(127.0.0.1:54508), state=TIMED_WAITING, group=TGRP-TestSolrCloudWithKerberos]
[junit4] > at java.lang.Thread.sleep(Native Method)
[junit4] > at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:994)
[junit4] > 3) Thread[id=5805, name=zkCallback-803-thread-1, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithKerberos]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
[junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
[junit4] > at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] > 4) Thread[id=5841, name=zkCallback-803-thread-3, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithKerberos]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
[junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
[junit4] > at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] > 5) Thread[id=5803, name=TEST-TestSolrCloudWithKerberos.testKerberizedSolr-seed#[7E178E8EB7F6CF8F]-EventThread, state=WAITING, group=TGRP-TestSolrCloudWithKerberos]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
[junit4] > at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4] > at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:494)
[junit4] > at __randomizedtesting.SeedInfo.seed([7E178E8EB7F6CF8F]:0)Throwable #3: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
[junit4] > 1) Thread[id=5806, name=zkCallback-803-thread-2, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithKerberos]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
[junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
[junit4] > at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] > 2) Thread[id=5802, name=TEST-TestSolrCloudWithKerberos.testKerberizedSolr-seed#[7E178E8EB7F6CF8F]-SendThread(127.0.0.1:54508), state=TIMED_WAITING, group=TGRP-TestSolrCloudWithKerberos]
[junit4] > at java.lang.Thread.sleep(Native Method)
[junit4] > at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:994)
[junit4] > 3) Thread[id=5805, name=zkCallback-803-thread-1, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithKerberos]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
[junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
[junit4] > at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] > 4) Thread[id=5841, name=zkCallback-803-thread-3, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithKerberos]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
[junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
[junit4] > at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] > at __randomizedtesting.SeedInfo.seed([7E178E8EB7F6CF8F]:0)
[junit4] Completed [143/502] on J1 in 52.93s, 1 test, 1 failure, 3 errors <<< FAILURES!
[...truncated 1092 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/build.xml:474: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/build.xml:61: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/extra-targets.xml:39: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build.xml:229: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/common-build.xml:512: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/lucene/common-build.xml:1415: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/lucene/common-build.xml:973: There were test failures: 502 suites, 1983 tests, 3 suite-level errors, 1 error, 1212 ignored (21 assumptions)
Total time: 37 minutes 53 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
No prior successful build to compare, so performing full copy of artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
[JENKINS] Lucene-Solr-Tests-trunk-Java8 - Build # 10 - Still
Failing
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-trunk-Java8/10/
1 tests failed.
FAILED: org.apache.solr.cloud.TestSolrCloudWithKerberos.testKerberizedSolr
Error Message:
No live SolrServers available to handle this request:[http://127.0.0.1:39003, http://127.0.0.1:51276, http://127.0.0.1:33817, http://127.0.0.1:40748, http://127.0.0.1:39669]
Stack Trace:
org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:39003, http://127.0.0.1:51276, http://127.0.0.1:33817, http://127.0.0.1:40748, http://127.0.0.1:39669]
at __randomizedtesting.SeedInfo.seed([7F80B71B49A3C81:ACE1DDB6FEFBFCB6]:0)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:355)
at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1086)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:856)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:799)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:135)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:152)
at org.apache.solr.cloud.TestSolrCloudWithKerberos.testKerberizedSolr(TestSolrCloudWithKerberos.java:159)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:960)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:935)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:39669: Expected mime type application/octet-stream but got text/html. <html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/>
<title>Error 500 </title>
</head>
<body>
<h2>HTTP ERROR: 500</h2>
<p>Problem accessing /admin/collections. Reason:
<pre> org.apache.solr.common.SolrException: Error during request authentication, </pre></p>
<hr /><i><small>Powered by Jetty://</small></i>
</body>
</html>
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:529)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:235)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:227)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:376)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:328)
... 47 more
Build Log:
[...truncated 10521 lines...]
[junit4] Suite: org.apache.solr.cloud.TestSolrCloudWithKerberos
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/init-core-data-001
[junit4] 2> 1478245 T8713 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
[junit4] 2> 1483607 T8713 oadsc.DefaultDirectoryService.showSecurityWarnings WARN You didn't change the admin password of directory service instance 'DefaultKrbServer'. Please update the admin password as soon as possible to prevent a possible security breach.
[junit4] 2> 1484334 T8713 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 2> 1484334 T8722 oasc.ZkTestServer$2$1.setClientPort client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 1484335 T8722 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 1484435 T8713 oasc.ZkTestServer.run start zk server on port:40569
[junit4] 2> 1484437 T8729 oascc.ConnectionManager.process WARN zkClient received AuthFailed
[junit4] 2> 1484447 T8732 oascc.ConnectionManager.process WARN zkClient received AuthFailed
[junit4] 2> 1484452 T8713 oasc.AbstractZkTestCase.putConfig put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 1484454 T8713 oasc.AbstractZkTestCase.putConfig put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4] 2> 1484456 T8713 oasc.AbstractZkTestCase.putConfig put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 1484458 T8713 oasc.AbstractZkTestCase.putConfig put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 1484459 T8713 oasc.AbstractZkTestCase.putConfig put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 1484461 T8713 oasc.AbstractZkTestCase.putConfig put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 1484462 T8713 oasc.AbstractZkTestCase.putConfig put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 1484464 T8713 oasc.AbstractZkTestCase.putConfig put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 1484465 T8713 oasc.AbstractZkTestCase.putConfig put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 1484467 T8713 oasc.AbstractZkTestCase.putConfig put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 1484468 T8713 oasc.AbstractZkTestCase.putConfig put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 1484472 T8736 oascc.ConnectionManager.process WARN zkClient received AuthFailed
[junit4] 2> 1484812 T8713 oas.SolrTestCaseJ4.writeCoreProperties Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/control-001/cores/collection1
[junit4] 2> 1484814 T8713 oejs.Server.doStart jetty-9.2.10.v20150310
[junit4] 2> 1484816 T8713 oejsh.ContextHandler.doStart Started o.e.j.s.ServletContextHandler@4436ea90{/,null,AVAILABLE}
[junit4] 2> 1484817 T8713 oejs.AbstractConnector.doStart Started ServerConnector@3a49cb8c{HTTP/1.1}{127.0.0.1:40748}
[junit4] 2> 1484817 T8713 oejs.Server.doStart Started @1486683ms
[junit4] 2> 1484817 T8713 oascse.JettySolrRunner$1.lifeCycleStarted Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/tempDir-002/control/data, hostContext=/, hostPort=40748, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/control-001/cores}
[junit4] 2> 1484818 T8713 oass.SolrDispatchFilter.init SolrDispatchFilter.init()sun.misc.Launcher$AppClassLoader@c387f44
[junit4] 2> 1484818 T8713 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/control-001/'
[junit4] 2> 1485974 T8713 oasc.SolrXmlConfig.fromFile Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/control-001/solr.xml
[junit4] 2> 1485980 T8713 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/control-001/cores
[junit4] 2> 1485981 T8713 oasc.CoreContainer.<init> New CoreContainer 222157298
[junit4] 2> 1485981 T8713 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/control-001/]
[junit4] 2> 1485981 T8713 oasc.CoreContainer.load loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/control-001/lib
[junit4] 2> 1485982 T8713 oasc.SolrResourceLoader.addToClassLoader WARN Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/control-001/lib).
[junit4] 2> 1485998 T8713 oashc.HttpShardHandlerFactory.init created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
[junit4] 2> 1486001 T8713 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 1486002 T8713 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 1486002 T8713 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 1486003 T8713 oasc.CoreContainer.load Node Name: 127.0.0.1
[junit4] 2> 1486003 T8713 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:40569/solr
[junit4] 2> 1486003 T8713 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 1486004 T8751 oascc.ConnectionManager.process WARN zkClient received AuthFailed
[junit4] 2> 1486015 T8755 n:127.0.0.1:40748_ oascc.ConnectionManager.process WARN zkClient received AuthFailed
[junit4] 2> 1486025 T8713 n:127.0.0.1:40748_ oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:40748_
[junit4] 2> 1486030 T8713 n:127.0.0.1:40748_ oasc.Overseer.close Overseer (id=null) closing
[junit4] 2> 1486031 T8713 n:127.0.0.1:40748_ oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:40748_
[junit4] 2> 1486033 T8713 n:127.0.0.1:40748_ oasc.Overseer.start Overseer (id=93883821892239364-127.0.0.1:40748_-n_0000000000) starting
[junit4] 2> 1486040 T8713 n:127.0.0.1:40748_ oasc.OverseerAutoReplicaFailoverThread.<init> Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
[junit4] 2> 1486041 T8758 n:127.0.0.1:40748_ oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 1486042 T8757 n:127.0.0.1:40748_ oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 1486045 T8713 n:127.0.0.1:40748_ oasc.CoreContainer.initializeAuthenticationPlugin Authentication plugin class obtained from ZK: org.apache.solr.security.KerberosPlugin
[junit4] 2> 1486054 T8713 n:127.0.0.1:40748_ oass.KerberosPlugin.init Params: {token.valid=30, kerberos.principal=HTTP/127.0.0.1, cookie.domain=127.0.0.1, kerberos.keytab=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/tempDir-001/minikdc/keytabs, cookie.path=/, type=kerberos, kerberos.name.rules=DEFAULT}
[junit4] 2> 1486075 T8713 n:127.0.0.1:40748_ oasc.CoreContainer.initializeAuthenticationPlugin Reconfiguring the shard handler factory and update shard handler.
[junit4] 2> 1486076 T8713 n:127.0.0.1:40748_ oashc.HttpShardHandlerFactory.reconfigureHttpClient Reconfiguring the default client with: org.apache.solr.client.solrj.impl.Krb5HttpClientConfigurer@5c575b56
[junit4] 2> 1486077 T8713 n:127.0.0.1:40748_ oascsi.Krb5HttpClientConfigurer.configure Setting up SPNego auth with config: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/tempDir-001/minikdc/jaas-client.conf
[junit4] 2> 1486080 T8713 n:127.0.0.1:40748_ oasu.UpdateShardHandler.reconfigureHttpClient Reconfiguring the default client with: org.apache.solr.client.solrj.impl.Krb5HttpClientConfigurer@5c575b56
[junit4] 2> 1486080 T8713 n:127.0.0.1:40748_ oascsi.Krb5HttpClientConfigurer.configure Setting up SPNego auth with config: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/tempDir-001/minikdc/jaas-client.conf
[junit4] 2> 1486083 T8713 n:127.0.0.1:40748_ oasc.CorePropertiesLocator.discover Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/control-001/cores
[junit4] 2> 1486084 T8713 n:127.0.0.1:40748_ oasc.CoreDescriptor.<init> CORE DESCRIPTOR: {name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, loadOnStartup=true, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/control-001/cores/collection1, collection=control_collection, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/control-001/cores/collection1/, coreNodeName=, dataDir=data/, shard=}
[junit4] 2> 1486084 T8713 n:127.0.0.1:40748_ oasc.CorePropertiesLocator.discoverUnder Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/control-001/cores/collection1/
[junit4] 2> 1486085 T8713 n:127.0.0.1:40748_ oasc.CorePropertiesLocator.discover Found 1 core definitions
[junit4] 2> 1486086 T8761 n:127.0.0.1:40748_ c:control_collection x:collection1 oasc.ZkController.publish publishing core=collection1 state=down collection=control_collection
[junit4] 2> 1486086 T8761 n:127.0.0.1:40748_ c:control_collection x:collection1 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 1486087 T8755 n:127.0.0.1:40748_ oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
[junit4] 2> 1486087 T8761 n:127.0.0.1:40748_ oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 1486088 T8757 n:127.0.0.1:40748_ oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"http://127.0.0.1:40748",
[junit4] 2> "node_name":"127.0.0.1:40748_",
[junit4] 2> "numShards":"1",
[junit4] 2> "state":"down",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"control_collection",
[junit4] 2> "operation":"state"} current state version: 0
[junit4] 2> 1486088 T8757 n:127.0.0.1:40748_ oasco.ReplicaMutator.updateState Update state numShards=1 message={
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"http://127.0.0.1:40748",
[junit4] 2> "node_name":"127.0.0.1:40748_",
[junit4] 2> "numShards":"1",
[junit4] 2> "state":"down",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"control_collection",
[junit4] 2> "operation":"state"}
[junit4] 2> 1486088 T8757 n:127.0.0.1:40748_ oasco.ClusterStateMutator.createCollection building a new cName: control_collection
[junit4] 2> 1486089 T8757 n:127.0.0.1:40748_ oasco.ReplicaMutator.updateState Assigning new node to shard shard=shard1
[junit4] 2> 1487087 T8761 n:127.0.0.1:40748_ oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 1487088 T8761 n:127.0.0.1:40748_ oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4] 2> 1487091 T8761 n:127.0.0.1:40748_ oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 1487092 T8761 n:127.0.0.1:40748_ oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/control-001/cores/collection1/'
[junit4] 2> 1487113 T8761 n:127.0.0.1:40748_ oasc.Config.<init> loaded config solrconfig.xml with version 0
[junit4] 2> 1487120 T8761 n:127.0.0.1:40748_ oasc.SolrConfig.refreshRequestParams current version of requestparams : -1
[junit4] 2> 1487139 T8761 n:127.0.0.1:40748_ oasc.SolrConfig.<init> Using Lucene MatchVersion: 6.0.0
[junit4] 2> 1487148 T8761 n:127.0.0.1:40748_ oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 1487149 T8761 n:127.0.0.1:40748_ oass.IndexSchema.readSchema Reading Solr Schema from /configs/conf1/schema.xml
[junit4] 2> 1487155 T8761 n:127.0.0.1:40748_ oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 1487275 T8761 n:127.0.0.1:40748_ oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 1487281 T8761 n:127.0.0.1:40748_ oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 1487283 T8761 n:127.0.0.1:40748_ oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 1487292 T8761 n:127.0.0.1:40748_ oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 1487294 T8761 n:127.0.0.1:40748_ oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 1487296 T8761 n:127.0.0.1:40748_ oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 1487296 T8761 n:127.0.0.1:40748_ oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 1487297 T8761 n:127.0.0.1:40748_ oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 1487297 T8761 n:127.0.0.1:40748_ oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 1487298 T8761 n:127.0.0.1:40748_ oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 1487298 T8761 n:127.0.0.1:40748_ oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 1487298 T8761 n:127.0.0.1:40748_ oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection control_collection
[junit4] 2> 1487298 T8761 n:127.0.0.1:40748_ x:collection1 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 1487299 T8761 n:127.0.0.1:40748_ x:collection1 oasc.SolrCore.<init> [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/control-001/cores/collection1/], dataDir=[null]
[junit4] 2> 1487299 T8761 n:127.0.0.1:40748_ x:collection1 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@17cfcf30
[junit4] 2> 1487299 T8761 n:127.0.0.1:40748_ x:collection1 oasc.CachingDirectoryFactory.get return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/control-001/cores/collection1/data
[junit4] 2> 1487300 T8761 n:127.0.0.1:40748_ x:collection1 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/control-001/cores/collection1/data/index/
[junit4] 2> 1487301 T8761 n:127.0.0.1:40748_ x:collection1 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
[junit4] 2> 1487301 T8761 n:127.0.0.1:40748_ x:collection1 oasc.CachingDirectoryFactory.get return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/control-001/cores/collection1/data/index
[junit4] 2> 1487301 T8761 n:127.0.0.1:40748_ x:collection1 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=33, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.3006284492664655]
[junit4] 2> 1487302 T8761 n:127.0.0.1:40748_ x:collection1 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@3229d82c lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@477df76e),segFN=segments_1,generation=1}
[junit4] 2> 1487302 T8761 n:127.0.0.1:40748_ x:collection1 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 1487306 T8761 n:127.0.0.1:40748_ x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 1487306 T8761 n:127.0.0.1:40748_ x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 1487306 T8761 n:127.0.0.1:40748_ x:collection1 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 1487307 T8761 n:127.0.0.1:40748_ x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 1487307 T8761 n:127.0.0.1:40748_ x:collection1 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 1487307 T8761 n:127.0.0.1:40748_ x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 1487307 T8761 n:127.0.0.1:40748_ x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 1487308 T8761 n:127.0.0.1:40748_ x:collection1 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 1487308 T8761 n:127.0.0.1:40748_ x:collection1 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 1487310 T8761 n:127.0.0.1:40748_ x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 1487311 T8761 n:127.0.0.1:40748_ x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 1487312 T8761 n:127.0.0.1:40748_ x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 1487313 T8761 n:127.0.0.1:40748_ x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 1487316 T8761 n:127.0.0.1:40748_ x:collection1 oasc.RequestHandlers.initHandlersFromConfig Registered paths: /admin/mbeans,standard,/update/csv,/update/json/docs,/admin/luke,/admin/segments,/get,/admin/system,/replication,/admin/properties,/config,/schema,/admin/plugins,/admin/logging,/update/json,/admin/threads,/admin/ping,/update,/admin/file
[junit4] 2> 1487317 T8761 n:127.0.0.1:40748_ x:collection1 oasc.SolrCore.initStatsCache Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
[junit4] 2> 1487322 T8761 n:127.0.0.1:40748_ x:collection1 oasu.UpdateHandler.<init> Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 1487323 T8761 n:127.0.0.1:40748_ x:collection1 oasu.UpdateLog.init Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 1487325 T8761 n:127.0.0.1:40748_ x:collection1 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 1487325 T8761 n:127.0.0.1:40748_ x:collection1 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 1487326 T8761 n:127.0.0.1:40748_ x:collection1 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=40, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
[junit4] 2> 1487327 T8761 n:127.0.0.1:40748_ x:collection1 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@3229d82c lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@477df76e),segFN=segments_1,generation=1}
[junit4] 2> 1487327 T8761 n:127.0.0.1:40748_ x:collection1 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 1487327 T8761 n:127.0.0.1:40748_ x:collection1 oass.SolrIndexSearcher.<init> Opening Searcher@6edfb126[collection1] main
[junit4] 2> 1487328 T8761 n:127.0.0.1:40748_ x:collection1 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
[junit4] 2> 1487329 T8761 n:127.0.0.1:40748_ x:collection1 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 1487329 T8761 n:127.0.0.1:40748_ x:collection1 oasr.RestManager.init Initializing RestManager with initArgs: {}
[junit4] 2> 1487329 T8761 n:127.0.0.1:40748_ x:collection1 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1487330 T8761 n:127.0.0.1:40748_ x:collection1 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json
[junit4] 2> 1487330 T8761 n:127.0.0.1:40748_ x:collection1 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1487330 T8761 n:127.0.0.1:40748_ x:collection1 oasr.RestManager.init Initializing 0 registered ManagedResources
[junit4] 2> 1487331 T8761 n:127.0.0.1:40748_ x:collection1 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4] 2> 1487331 T8763 n:127.0.0.1:40748_ x:collection1 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6edfb126[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1487332 T8761 n:127.0.0.1:40748_ x:collection1 oasc.ZkController.getConfDirListeners watch zkdir /configs/conf1
[junit4] 2> 1487332 T8761 n:127.0.0.1:40748_ x:collection1 oasu.UpdateLog.onFirstSearcher On first searcher opened, looking up max value of version field
[junit4] 2> 1487332 T8761 n:127.0.0.1:40748_ x:collection1 oasu.VersionInfo.getMaxVersionFromIndex Refreshing highest value of _version_ for 65536 version buckets from index
[junit4] 2> 1487333 T8761 n:127.0.0.1:40748_ x:collection1 oasu.VersionInfo.getMaxVersionFromIndex No terms found for _version_, cannot seed version bucket highest value from index
[junit4] 2> 1487333 T8761 n:127.0.0.1:40748_ x:collection1 oasu.UpdateLog.seedBucketsWithHighestVersion Could not find max version in index or recent updates, using new clock 1502141153418412032
[junit4] 2> 1487335 T8761 n:127.0.0.1:40748_ x:collection1 oasu.UpdateLog.seedBucketsWithHighestVersion Took 3 ms to seed version buckets with highest version 1502141153418412032
[junit4] 2> 1487336 T8761 n:127.0.0.1:40748_ x:collection1 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 1487341 T8766 n:127.0.0.1:40748_ c:control_collection s:shard1 x:collection1 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:40748 collection:control_collection shard:shard1
[junit4] 2> 1487341 T8713 n:127.0.0.1:40748_ oascsi.Krb5HttpClientConfigurer.configure Setting up SPNego auth with config: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/tempDir-001/minikdc/jaas-client.conf
[junit4] 2> 1487342 T8713 n:127.0.0.1:40748_ oass.SolrDispatchFilter.init user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2
[junit4] 2> 1487342 T8713 n:127.0.0.1:40748_ oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 1487343 T8713 oascsi.Krb5HttpClientConfigurer.configure Setting up SPNego auth with config: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/tempDir-001/minikdc/jaas-client.conf
[junit4] 2> 1487344 T8713 oascsi.Krb5HttpClientConfigurer.configure Setting up SPNego auth with config: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/tempDir-001/minikdc/jaas-client.conf
[junit4] 2> 1487347 T8769 oascc.ConnectionManager.process WARN zkClient received AuthFailed
[junit4] 2> 1487352 T8766 n:127.0.0.1:40748_ c:control_collection s:shard1 x:collection1 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 1487354 T8755 n:127.0.0.1:40748_ oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
[junit4] 2> 1487354 T8766 n:127.0.0.1:40748_ c:control_collection s:shard1 x:collection1 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 1487355 T8757 n:127.0.0.1:40748_ oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"leader",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"control_collection"} current state version: 1
[junit4] 2> 1487355 T8766 n:127.0.0.1:40748_ c:control_collection s:shard1 x:collection1 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C1209 name=collection1 org.apache.solr.core.SolrCore@c68063c url=http://127.0.0.1:40748/collection1 node=127.0.0.1:40748_ C1209_STATE=coll:control_collection core:collection1 props:{core=collection1, base_url=http://127.0.0.1:40748, node_name=127.0.0.1:40748_, state=down}
[junit4] 2> 1487356 T8766 n:127.0.0.1:40748_ c:control_collection s:shard1 x:collection1 C1209 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:40748/collection1/
[junit4] 2> 1487356 T8766 n:127.0.0.1:40748_ c:control_collection s:shard1 x:collection1 C1209 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 1487357 T8766 n:127.0.0.1:40748_ c:control_collection s:shard1 x:collection1 C1209 oasc.SyncStrategy.syncToMe http://127.0.0.1:40748/collection1/ has no replicas
[junit4] 2> 1487357 T8766 n:127.0.0.1:40748_ c:control_collection s:shard1 x:collection1 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:40748/collection1/ shard1
[junit4] 2> 1487360 T8755 n:127.0.0.1:40748_ oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
[junit4] 2> 1487361 T8713 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 1487361 T8757 n:127.0.0.1:40748_ oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"leader",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"control_collection",
[junit4] 2> "base_url":"http://127.0.0.1:40748",
[junit4] 2> "core":"collection1",
[junit4] 2> "state":"active"} current state version: 1
[junit4] 2> 1487512 T8766 n:127.0.0.1:40748_ c:control_collection s:shard1 x:collection1 oasc.ZkController.register We are http://127.0.0.1:40748/collection1/ and leader is http://127.0.0.1:40748/collection1/
[junit4] 2> 1487512 T8766 n:127.0.0.1:40748_ c:control_collection s:shard1 x:collection1 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:40748
[junit4] 2> 1487513 T8766 n:127.0.0.1:40748_ c:control_collection s:shard1 x:collection1 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 1487513 T8766 n:127.0.0.1:40748_ c:control_collection s:shard1 x:collection1 oasc.ZkController.publish publishing core=collection1 state=active collection=control_collection
[junit4] 2> 1487513 T8766 n:127.0.0.1:40748_ c:control_collection s:shard1 x:collection1 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 1487514 T8755 n:127.0.0.1:40748_ oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
[junit4] 2> 1487515 T8757 n:127.0.0.1:40748_ oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
[junit4] 2> "core":"collection1",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"http://127.0.0.1:40748",
[junit4] 2> "node_name":"127.0.0.1:40748_",
[junit4] 2> "numShards":"2",
[junit4] 2> "state":"active",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"control_collection",
[junit4] 2> "operation":"state"} current state version: 2
[junit4] 2> 1487516 T8757 n:127.0.0.1:40748_ oasco.ReplicaMutator.updateState Update state numShards=2 message={
[junit4] 2> "core":"collection1",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"http://127.0.0.1:40748",
[junit4] 2> "node_name":"127.0.0.1:40748_",
[junit4] 2> "numShards":"2",
[junit4] 2> "state":"active",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"control_collection",
[junit4] 2> "operation":"state"}
[junit4] 2> 1487719 T8713 oas.SolrTestCaseJ4.writeCoreProperties Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/shard-1-001/cores/collection1
[junit4] 2> 1487721 T8713 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/shard-1-001
[junit4] 2> 1487722 T8713 oejs.Server.doStart jetty-9.2.10.v20150310
[junit4] 2> 1487727 T8713 oejsh.ContextHandler.doStart Started o.e.j.s.ServletContextHandler@2eab3622{/,null,AVAILABLE}
[junit4] 2> 1487728 T8713 oejs.AbstractConnector.doStart Started ServerConnector@31002725{HTTP/1.1}{127.0.0.1:39003}
[junit4] 2> 1487729 T8713 oejs.Server.doStart Started @1489595ms
[junit4] 2> 1487729 T8713 oascse.JettySolrRunner$1.lifeCycleStarted Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/tempDir-002/jetty1, solrconfig=solrconfig.xml, hostContext=/, hostPort=39003, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/shard-1-001/cores}
[junit4] 2> 1487729 T8713 oascsi.Krb5HttpClientConfigurer.configure Setting up SPNego auth with config: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/tempDir-001/minikdc/jaas-client.conf
[junit4] 2> 1487730 T8713 oass.SolrDispatchFilter.init SolrDispatchFilter.init()sun.misc.Launcher$AppClassLoader@c387f44
[junit4] 2> 1487730 T8713 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/shard-1-001/'
[junit4] 2> 1487745 T8713 oasc.SolrXmlConfig.fromFile Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/shard-1-001/solr.xml
[junit4] 2> 1487750 T8713 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/shard-1-001/cores
[junit4] 2> 1487751 T8713 oasc.CoreContainer.<init> New CoreContainer 454062642
[junit4] 2> 1487751 T8713 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/shard-1-001/]
[junit4] 2> 1487751 T8713 oasc.CoreContainer.load loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/shard-1-001/lib
[junit4] 2> 1487751 T8713 oasc.SolrResourceLoader.addToClassLoader WARN Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/shard-1-001/lib).
[junit4] 2> 1487758 T8713 oashc.HttpShardHandlerFactory.init created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
[junit4] 2> 1487759 T8713 oascsi.Krb5HttpClientConfigurer.configure Setting up SPNego auth with config: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/tempDir-001/minikdc/jaas-client.conf
[junit4] 2> 1487760 T8713 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 1487761 T8713 oascsi.Krb5HttpClientConfigurer.configure Setting up SPNego auth with config: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/tempDir-001/minikdc/jaas-client.conf
[junit4] 2> 1487761 T8713 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 1487761 T8713 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 1487762 T8713 oasc.CoreContainer.load Node Name: 127.0.0.1
[junit4] 2> 1487762 T8713 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:40569/solr
[junit4] 2> 1487762 T8713 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 1487764 T8783 oascc.ConnectionManager.process WARN zkClient received AuthFailed
[junit4] 2> 1487767 T8786 n:127.0.0.1:39003_ oascc.ConnectionManager.process WARN zkClient received AuthFailed
[junit4] 2> 1488783 T8713 n:127.0.0.1:39003_ oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:39003_
[junit4] 2> 1488786 T8713 n:127.0.0.1:39003_ oasc.Overseer.close Overseer (id=null) closing
[junit4] 2> 1488788 T8713 n:127.0.0.1:39003_ oasc.CoreContainer.initializeAuthenticationPlugin Authentication plugin class obtained from ZK: org.apache.solr.security.KerberosPlugin
[junit4] 2> 1488788 T8713 n:127.0.0.1:39003_ oass.KerberosPlugin.init Params: {token.valid=30, kerberos.principal=HTTP/127.0.0.1, cookie.domain=127.0.0.1, kerberos.keytab=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/tempDir-001/minikdc/keytabs, cookie.path=/, type=kerberos, kerberos.name.rules=DEFAULT}
[junit4] 2> 1488790 T8713 n:127.0.0.1:39003_ oasc.CoreContainer.initializeAuthenticationPlugin Reconfiguring the shard handler factory and update shard handler.
[junit4] 2> 1488790 T8713 n:127.0.0.1:39003_ oashc.HttpShardHandlerFactory.reconfigureHttpClient Reconfiguring the default client with: org.apache.solr.client.solrj.impl.Krb5HttpClientConfigurer@4334d2
[junit4] 2> 1488791 T8713 n:127.0.0.1:39003_ oascsi.Krb5HttpClientConfigurer.configure Setting up SPNego auth with config: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/tempDir-001/minikdc/jaas-client.conf
[junit4] 2> 1488791 T8713 n:127.0.0.1:39003_ oasu.UpdateShardHandler.reconfigureHttpClient Reconfiguring the default client with: org.apache.solr.client.solrj.impl.Krb5HttpClientConfigurer@4334d2
[junit4] 2> 1488791 T8713 n:127.0.0.1:39003_ oascsi.Krb5HttpClientConfigurer.configure Setting up SPNego auth with config: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/tempDir-001/minikdc/jaas-client.conf
[junit4] 2> 1488793 T8713 n:127.0.0.1:39003_ oasc.CorePropertiesLocator.discover Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/shard-1-001/cores
[junit4] 2> 1488794 T8713 n:127.0.0.1:39003_ oasc.CoreDescriptor.<init> CORE DESCRIPTOR: {name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, loadOnStartup=true, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/shard-1-001/cores/collection1, collection=collection1, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/shard-1-001/cores/collection1/, coreNodeName=, dataDir=data/, shard=}
[junit4] 2> 1488794 T8713 n:127.0.0.1:39003_ oasc.CorePropertiesLocator.discoverUnder Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/shard-1-001/cores/collection1/
[junit4] 2> 1488795 T8713 n:127.0.0.1:39003_ oasc.CorePropertiesLocator.discover Found 1 core definitions
[junit4] 2> 1488799 T8788 n:127.0.0.1:39003_ c:collection1 x:collection1 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
[junit4] 2> 1488799 T8788 n:127.0.0.1:39003_ c:collection1 x:collection1 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 1488800 T8788 n:127.0.0.1:39003_ oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 1488800 T8755 n:127.0.0.1:40748_ oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
[junit4] 2> 1488801 T8757 n:127.0.0.1:40748_ oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"http://127.0.0.1:39003",
[junit4] 2> "node_name":"127.0.0.1:39003_",
[junit4] 2> "numShards":"2",
[junit4] 2> "state":"down",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "operation":"state"} current state version: 3
[junit4] 2> 1488801 T8757 n:127.0.0.1:40748_ oasco.ReplicaMutator.updateState Update state numShards=2 message={
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"http://127.0.0.1:39003",
[junit4] 2> "node_name":"127.0.0.1:39003_",
[junit4] 2> "numShards":"2",
[junit4] 2> "state":"down",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "operation":"state"}
[junit4] 2> 1488802 T8757 n:127.0.0.1:40748_ oasco.ClusterStateMutator.createCollection building a new cName: collection1
[junit4] 2> 1488802 T8757 n:127.0.0.1:40748_ oasco.ReplicaMutator.updateState Assigning new node to shard shard=shard2
[junit4] 2> 1489800 T8788 n:127.0.0.1:39003_ oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 1489801 T8788 n:127.0.0.1:39003_ oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 1489801 T8788 n:127.0.0.1:39003_ oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 1489802 T8788 n:127.0.0.1:39003_ oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/shard-1-001/cores/collection1/'
[junit4] 2> 1489814 T8788 n:127.0.0.1:39003_ oasc.Config.<init> loaded config solrconfig.xml with version 0
[junit4] 2> 1489820 T8788 n:127.0.0.1:39003_ oasc.SolrConfig.refreshRequestParams current version of requestparams : -1
[junit4] 2> 1489826 T8788 n:127.0.0.1:39003_ oasc.SolrConfig.<init> Using Lucene MatchVersion: 6.0.0
[junit4] 2> 1489841 T8788 n:127.0.0.1:39003_ oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 1489842 T8788 n:127.0.0.1:39003_ oass.IndexSchema.readSchema Reading Solr Schema from /configs/conf1/schema.xml
[junit4] 2> 1489854 T8788 n:127.0.0.1:39003_ oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 1490019 T8788 n:127.0.0.1:39003_ oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 1490025 T8788 n:127.0.0.1:39003_ oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 1490026 T8788 n:127.0.0.1:39003_ oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 1490034 T8788 n:127.0.0.1:39003_ oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 1490036 T8788 n:127.0.0.1:39003_ oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 1490038 T8788 n:127.0.0.1:39003_ oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 1490039 T8788 n:127.0.0.1:39003_ oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 1490039 T8788 n:127.0.0.1:39003_ oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 1490039 T8788 n:127.0.0.1:39003_ oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 1490040 T8788 n:127.0.0.1:39003_ oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 1490040 T8788 n:127.0.0.1:39003_ oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 1490040 T8788 n:127.0.0.1:39003_ oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection collection1
[junit4] 2> 1490040 T8788 n:127.0.0.1:39003_ x:collection1 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 1490041 T8788 n:127.0.0.1:39003_ x:collection1 oasc.SolrCore.<init> [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/shard-1-001/cores/collection1/], dataDir=[null]
[junit4] 2> 1490041 T8788 n:127.0.0.1:39003_ x:collection1 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@17cfcf30
[junit4] 2> 1490041 T8788 n:127.0.0.1:39003_ x:collection1 oasc.CachingDirectoryFactory.get return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/shard-1-001/cores/collection1/data
[junit4] 2> 1490042 T8788 n:127.0.0.1:39003_ x:collection1 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/shard-1-001/cores/collection1/data/index/
[junit4] 2> 1490042 T8788 n:127.0.0.1:39003_ x:collection1 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/shard-1-001/cores/collection1/data/index' doesn't exist. Creating new index...
[junit4] 2> 1490042 T8788 n:127.0.0.1:39003_ x:collection1 oasc.CachingDirectoryFactory.get return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/shard-1-001/cores/collection1/data/index
[junit4] 2> 1490043 T8788 n:127.0.0.1:39003_ x:collection1 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=33, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.3006284492664655]
[junit4] 2> 1490043 T8788 n:127.0.0.1:39003_ x:collection1 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@7211656a lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@293e526a),segFN=segments_1,generation=1}
[junit4] 2> 1490043 T8788 n:127.0.0.1:39003_ x:collection1 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 1490047 T8788 n:127.0.0.1:39003_ x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 1490048 T8788 n:127.0.0.1:39003_ x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 1490048 T8788 n:127.0.0.1:39003_ x:collection1 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 1490048 T8788 n:127.0.0.1:39003_ x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 1490048 T8788 n:127.0.0.1:39003_ x:collection1 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 1490048 T8788 n:127.0.0.1:39003_ x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 1490049 T8788 n:127.0.0.1:39003_ x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 1490049 T8788 n:127.0.0.1:39003_ x:collection1 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 1490049 T8788 n:127.0.0.1:39003_ x:collection1 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 1490051 T8788 n:127.0.0.1:39003_ x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 1490052 T8788 n:127.0.0.1:39003_ x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 1490053 T8788 n:127.0.0.1:39003_ x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 1490054 T8788 n:127.0.0.1:39003_ x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 1490058 T8788 n:127.0.0.1:39003_ x:collection1 oasc.RequestHandlers.initHandlersFromConfig Registered paths: /admin/mbeans,standard,/update/csv,/update/json/docs,/admin/luke,/admin/segments,/get,/admin/system,/replication,/admin/properties,/config,/schema,/admin/plugins,/admin/logging,/update/json,/admin/threads,/admin/ping,/update,/admin/file
[junit4] 2> 1490059 T8788 n:127.0.0.1:39003_ x:collection1 oasc.SolrCore.initStatsCache Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
[junit4] 2> 1490060 T8788 n:127.0.0.1:39003_ x:collection1 oasu.UpdateHandler.<init> Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 1490060 T8788 n:127.0.0.1:39003_ x:collection1 oasu.UpdateLog.init Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 1490061 T8788 n:127.0.0.1:39003_ x:collection1 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 1490061 T8788 n:127.0.0.1:39003_ x:collection1 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 1490062 T8788 n:127.0.0.1:39003_ x:collection1 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=40, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
[junit4] 2> 1490062 T8788 n:127.0.0.1:39003_ x:collection1 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@7211656a lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@293e526a),segFN=segments_1,generation=1}
[junit4] 2> 1490062 T8788 n:127.0.0.1:39003_ x:collection1 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 1490063 T8788 n:127.0.0.1:39003_ x:collection1 oass.SolrIndexSearcher.<init> Opening Searcher@365f21f0[collection1] main
[junit4] 2> 1490070 T8788 n:127.0.0.1:39003_ x:collection1 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
[junit4] 2> 1490071 T8788 n:127.0.0.1:39003_ x:collection1 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 1490071 T8788 n:127.0.0.1:39003_ x:collection1 oasr.RestManager.init Initializing RestManager with initArgs: {}
[junit4] 2> 1490071 T8788 n:127.0.0.1:39003_ x:collection1 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1490072 T8788 n:127.0.0.1:39003_ x:collection1 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json
[junit4] 2> 1490072 T8788 n:127.0.0.1:39003_ x:collection1 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1490072 T8788 n:127.0.0.1:39003_ x:collection1 oasr.RestManager.init Initializing 0 registered ManagedResources
[junit4] 2> 1490072 T8788 n:127.0.0.1:39003_ x:collection1 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4] 2> 1490073 T8788 n:127.0.0.1:39003_ x:collection1 oasc.ZkController.getConfDirListeners watch zkdir /configs/conf1
[junit4] 2> 1490074 T8789 n:127.0.0.1:39003_ x:collection1 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@365f21f0[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1490074 T8788 n:127.0.0.1:39003_ x:collection1 oasu.UpdateLog.onFirstSearcher On first searcher opened, looking up max value of version field
[junit4] 2> 1490074 T8788 n:127.0.0.1:39003_ x:collection1 oasu.VersionInfo.getMaxVersionFromIndex Refreshing highest value of _version_ for 65536 version buckets from index
[junit4] 2> 1490074 T8788 n:127.0.0.1:39003_ x:collection1 oasu.VersionInfo.getMaxVersionFromIndex No terms found for _version_, cannot seed version bucket highest value from index
[junit4] 2> 1490074 T8788 n:127.0.0.1:39003_ x:collection1 oasu.UpdateLog.seedBucketsWithHighestVersion Could not find max version in index or recent updates, using new clock 1502141156292558848
[junit4] 2> 1490077 T8788 n:127.0.0.1:39003_ x:collection1 oasu.UpdateLog.seedBucketsWithHighestVersion Took 3 ms to seed version buckets with highest version 1502141156292558848
[junit4] 2> 1490077 T8788 n:127.0.0.1:39003_ x:collection1 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 1490078 T8792 n:127.0.0.1:39003_ c:collection1 s:shard2 x:collection1 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:39003 collection:collection1 shard:shard2
[junit4] 2> 1490078 T8713 n:127.0.0.1:39003_ oascsi.Krb5HttpClientConfigurer.configure Setting up SPNego auth with config: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/tempDir-001/minikdc/jaas-client.conf
[junit4] 2> 1490079 T8713 n:127.0.0.1:39003_ oass.SolrDispatchFilter.init user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2
[junit4] 2> 1490079 T8713 n:127.0.0.1:39003_ oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 1490080 T8713 oascsi.Krb5HttpClientConfigurer.configure Setting up SPNego auth with config: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/tempDir-001/minikdc/jaas-client.conf
[junit4] 2> 1490082 T8792 n:127.0.0.1:39003_ c:collection1 s:shard2 x:collection1 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard2
[junit4] 2> 1490084 T8755 n:127.0.0.1:40748_ oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
[junit4] 2> 1490084 T8792 n:127.0.0.1:39003_ c:collection1 s:shard2 x:collection1 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 1490084 T8792 n:127.0.0.1:39003_ c:collection1 s:shard2 x:collection1 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C1210 name=collection1 org.apache.solr.core.SolrCore@680e69fb url=http://127.0.0.1:39003/collection1 node=127.0.0.1:39003_ C1210_STATE=coll:collection1 core:collection1 props:{core=collection1, base_url=http://127.0.0.1:39003, node_name=127.0.0.1:39003_, state=down}
[junit4] 2> 1490084 T8792 n:127.0.0.1:39003_ c:collection1 s:shard2 x:collection1 C1210 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:39003/collection1/
[junit4] 2> 1490084 T8757 n:127.0.0.1:40748_ oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"leader",
[junit4] 2> "shard":"shard2",
[junit4] 2> "collection":"collection1"} current state version: 4
[junit4] 2> 1490085 T8792 n:127.0.0.1:39003_ c:collection1 s:shard2 x:collection1 C1210 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 1490085 T8792 n:127.0.0.1:39003_ c:collection1 s:shard2 x:collection1 C1210 oasc.SyncStrategy.syncToMe http://127.0.0.1:39003/collection1/ has no replicas
[junit4] 2> 1490085 T8792 n:127.0.0.1:39003_ c:collection1 s:shard2 x:collection1 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:39003/collection1/ shard2
[junit4] 2> 1490088 T8755 n:127.0.0.1:40748_ oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
[junit4] 2> 1490089 T8757 n:127.0.0.1:40748_ oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"leader",
[junit4] 2> "shard":"shard2",
[junit4] 2> "collection":"collection1",
[junit4] 2> "base_url":"http://127.0.0.1:39003",
[junit4] 2> "core":"collection1",
[junit4] 2> "state":"active"} current state version: 4
[junit4] 2> 1490240 T8792 n:127.0.0.1:39003_ c:collection1 s:shard2 x:collection1 oasc.ZkController.register We are http://127.0.0.1:39003/collection1/ and leader is http://127.0.0.1:39003/collection1/
[junit4] 2> 1490240 T8792 n:127.0.0.1:39003_ c:collection1 s:shard2 x:collection1 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:39003
[junit4] 2> 1490240 T8792 n:127.0.0.1:39003_ c:collection1 s:shard2 x:collection1 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 1490240 T8792 n:127.0.0.1:39003_ c:collection1 s:shard2 x:collection1 oasc.ZkController.publish publishing core=collection1 state=active collection=collection1
[junit4] 2> 1490241 T8792 n:127.0.0.1:39003_ c:collection1 s:shard2 x:collection1 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 1490241 T8755 n:127.0.0.1:40748_ oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
[junit4] 2> 1490242 T8757 n:127.0.0.1:40748_ oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
[junit4] 2> "core":"collection1",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"http://127.0.0.1:39003",
[junit4] 2> "node_name":"127.0.0.1:39003_",
[junit4] 2> "numShards":"2",
[junit4] 2> "state":"active",
[junit4] 2> "shard":"shard2",
[junit4] 2> "collection":"collection1",
[junit4] 2> "operation":"state"} current state version: 5
[junit4] 2> 1490243 T8757 n:127.0.0.1:40748_ oasco.ReplicaMutator.updateState Update state numShards=2 message={
[junit4] 2> "core":"collection1",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"http://127.0.0.1:39003",
[junit4] 2> "node_name":"127.0.0.1:39003_",
[junit4] 2> "numShards":"2",
[junit4] 2> "state":"active",
[junit4] 2> "shard":"shard2",
[junit4] 2> "collection":"collection1",
[junit4] 2> "operation":"state"}
[junit4] 2> 1490432 T8713 oas.SolrTestCaseJ4.writeCoreProperties Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/shard-2-001/cores/collection1
[junit4] 2> 1490433 T8713 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 2 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/shard-2-001
[junit4] 2> 1490434 T8713 oejs.Server.doStart jetty-9.2.10.v20150310
[junit4] 2> 1490436 T8713 oejsh.ContextHandler.doStart Started o.e.j.s.ServletContextHandler@5f1593e7{/,null,AVAILABLE}
[junit4] 2> 1490437 T8713 oejs.AbstractConnector.doStart Started ServerConnector@4a36cc46{HTTP/1.1}{127.0.0.1:33817}
[junit4] 2> 1490437 T8713 oejs.Server.doStart Started @1492303ms
[junit4] 2> 1490438 T8713 oascse.JettySolrRunner$1.lifeCycleStarted Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/tempDir-002/jetty2, solrconfig=solrconfig.xml, hostContext=/, hostPort=33817, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/shard-2-001/cores}
[junit4] 2> 1490438 T8713 oascsi.Krb5HttpClientConfigurer.configure Setting up SPNego auth with config: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/tempDir-001/minikdc/jaas-client.conf
[junit4] 2> 1490438 T8713 oass.SolrDispatchFilter.init SolrDispatchFilter.init()sun.misc.Launcher$AppClassLoader@c387f44
[junit4] 2> 1490439 T8713 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/shard-2-001/'
[junit4] 2> 1490454 T8713 oasc.SolrXmlConfig.fromFile Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/shard-2-001/solr.xml
[junit4] 2> 1490475 T8713 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/shard-2-001/cores
[junit4] 2> 1490476 T8713 oasc.CoreContainer.<init> New CoreContainer 6697740
[junit4] 2> 1490477 T8713 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/shard-2-001/]
[junit4] 2> 1490477 T8713 oasc.CoreContainer.load loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/shard-2-001/lib
[junit4] 2> 1490479 T8713 oasc.SolrResourceLoader.addToClassLoader WARN Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/shard-2-001/lib).
[junit4] 2> 1490486 T8713 oashc.HttpShardHandlerFactory.init created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
[junit4] 2> 1490487 T8713 oascsi.Krb5HttpClientConfigurer.configure Setting up SPNego auth with config: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/tempDir-001/minikdc/jaas-client.conf
[junit4] 2> 1490488 T8713 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 1490488 T8713 oascsi.Krb5HttpClientConfigurer.configure Setting up SPNego auth with config: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/tempDir-001/minikdc/jaas-client.conf
[junit4] 2> 1490489 T8713 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 1490489 T8713 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 1490489 T8713 oasc.CoreContainer.load Node Name: 127.0.0.1
[junit4] 2> 1490489 T8713 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:40569/solr
[junit4] 2> 1490490 T8713 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 1490495 T8806 oascc.ConnectionManager.process WARN zkClient received AuthFailed
[junit4] 2> 1490508 T8810 n:127.0.0.1:33817_ oascc.ConnectionManager.process WARN zkClient received AuthFailed
[junit4] 2> 1491530 T8713 n:127.0.0.1:33817_ oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:33817_
[junit4] 2> 1491534 T8713 n:127.0.0.1:33817_ oasc.Overseer.close Overseer (id=null) closing
[junit4] 2> 1491535 T8713 n:127.0.0.1:33817_ oasc.CoreContainer.initializeAuthenticationPlugin Authentication plugin class obtained from ZK: org.apache.solr.security.KerberosPlugin
[junit4] 2> 1491536 T8713 n:127.0.0.1:33817_ oass.KerberosPlugin.init Params: {token.valid=30, kerberos.principal=HTTP/127.0.0.1, cookie.domain=127.0.0.1, kerberos.keytab=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/tempDir-001/minikdc/keytabs, cookie.path=/, type=kerberos, kerberos.name.rules=DEFAULT}
[junit4] 2> 1491538 T8713 n:127.0.0.1:33817_ oasc.CoreContainer.initializeAuthenticationPlugin Reconfiguring the shard handler factory and update shard handler.
[junit4] 2> 1491538 T8713 n:127.0.0.1:33817_ oashc.HttpShardHandlerFactory.reconfigureHttpClient Reconfiguring the default client with: org.apache.solr.client.solrj.impl.Krb5HttpClientConfigurer@3c3729d7
[junit4] 2> 1491539 T8713 n:127.0.0.1:33817_ oascsi.Krb5HttpClientConfigurer.configure Setting up SPNego auth with config: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/tempDir-001/minikdc/jaas-client.conf
[junit4] 2> 1491539 T8713 n:127.0.0.1:33817_ oasu.UpdateShardHandler.reconfigureHttpClient Reconfiguring the default client with: org.apache.solr.client.solrj.impl.Krb5HttpClientConfigurer@3c3729d7
[junit4] 2> 1491539 T8713 n:127.0.0.1:33817_ oascsi.Krb5HttpClientConfigurer.configure Setting up SPNego auth with config: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/tempDir-001/minikdc/jaas-client.conf
[junit4] 2> 1491540 T8713 n:127.0.0.1:33817_ oasc.CorePropertiesLocator.discover Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/shard-2-001/cores
[junit4] 2> 1491541 T8713 n:127.0.0.1:33817_ oasc.CoreDescriptor.<init> CORE DESCRIPTOR: {name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, loadOnStartup=true, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/shard-2-001/cores/collection1, collection=collection1, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/shard-2-001/cores/collection1/, coreNodeName=, dataDir=data/, shard=}
[junit4] 2> 1491542 T8713 n:127.0.0.1:33817_ oasc.CorePropertiesLocator.discoverUnder Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/shard-2-001/cores/collection1/
[junit4] 2> 1491542 T8713 n:127.0.0.1:33817_ oasc.CorePropertiesLocator.discover Found 1 core definitions
[junit4] 2> 1491543 T8813 n:127.0.0.1:33817_ c:collection1 x:collection1 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
[junit4] 2> 1491543 T8813 n:127.0.0.1:33817_ c:collection1 x:collection1 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 1491544 T8755 n:127.0.0.1:40748_ oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
[junit4] 2> 1491544 T8813 n:127.0.0.1:33817_ oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 1491545 T8757 n:127.0.0.1:40748_ oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"http://127.0.0.1:33817",
[junit4] 2> "node_name":"127.0.0.1:33817_",
[junit4] 2> "numShards":"2",
[junit4] 2> "state":"down",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "operation":"state"} current state version: 6
[junit4] 2> 1491545 T8757 n:127.0.0.1:40748_ oasco.ReplicaMutator.updateState Update state numShards=2 message={
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"http://127.0.0.1:33817",
[junit4] 2> "node_name":"127.0.0.1:33817_",
[junit4] 2> "numShards":"2",
[junit4] 2> "state":"down",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "operation":"state"}
[junit4] 2> 1491545 T8757 n:127.0.0.1:40748_ oasco.ReplicaMutator.updateState Collection already exists with numShards=2
[junit4] 2> 1491546 T8757 n:127.0.0.1:40748_ oasco.ReplicaMutator.updateState Assigning new node to shard shard=shard1
[junit4] 2> 1492544 T8813 n:127.0.0.1:33817_ oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 1492545 T8813 n:127.0.0.1:33817_ oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 1492545 T8813 n:127.0.0.1:33817_ oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 1492546 T8813 n:127.0.0.1:33817_ oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/shard-2-001/cores/collection1/'
[junit4] 2> 1492558 T8813 n:127.0.0.1:33817_ oasc.Config.<init> loaded config solrconfig.xml with version 0
[junit4] 2> 1492562 T8813 n:127.0.0.1:33817_ oasc.SolrConfig.refreshRequestParams current version of requestparams : -1
[junit4] 2> 1492566 T8813 n:127.0.0.1:33817_ oasc.SolrConfig.<init> Using Lucene MatchVersion: 6.0.0
[junit4] 2> 1492575 T8813 n:127.0.0.1:33817_ oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 1492576 T8813 n:127.0.0.1:33817_ oass.IndexSchema.readSchema Reading Solr Schema from /configs/conf1/schema.xml
[junit4] 2> 1492581 T8813 n:127.0.0.1:33817_ oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 1492685 T8813 n:127.0.0.1:33817_ oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 1492690 T8813 n:127.0.0.1:33817_ oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 1492692 T8813 n:127.0.0.1:33817_ oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 1492700 T8813 n:127.0.0.1:33817_ oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 1492702 T8813 n:127.0.0.1:33817_ oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 1492703 T8813 n:127.0.0.1:33817_ oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 1492704 T8813 n:127.0.0.1:33817_ oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 1492704 T8813 n:127.0.0.1:33817_ oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 1492705 T8813 n:127.0.0.1:33817_ oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 1492705 T8813 n:127.0.0.1:33817_ oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 1492706 T8813 n:127.0.0.1:33817_ oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 1492706 T8813 n:127.0.0.1:33817_ oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection collection1
[junit4] 2> 1492706 T8813 n:127.0.0.1:33817_ x:collection1 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 1492706 T8813 n:127.0.0.1:33817_ x:collection1 oasc.SolrCore.<init> [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/shard-2-001/cores/collection1/], dataDir=[null]
[junit4] 2> 1492707 T8813 n:127.0.0.1:33817_ x:collection1 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@17cfcf30
[junit4] 2> 1492707 T8813 n:127.0.0.1:33817_ x:collection1 oasc.CachingDirectoryFactory.get return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/shard-2-001/cores/collection1/data
[junit4] 2> 1492707 T8813 n:127.0.0.1:33817_ x:collection1 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/shard-2-001/cores/collection1/data/index/
[junit4] 2> 1492707 T8813 n:127.0.0.1:33817_ x:collection1 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/shard-2-001/cores/collection1/data/index' doesn't exist. Creating new index...
[junit4] 2> 1492708 T8813 n:127.0.0.1:33817_ x:collection1 oasc.CachingDirectoryFactory.get return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/shard-2-001/cores/collection1/data/index
[junit4] 2> 1492708 T8813 n:127.0.0.1:33817_ x:collection1 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=33, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.3006284492664655]
[junit4] 2> 1492708 T8813 n:127.0.0.1:33817_ x:collection1 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@d394c40 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@abd78ba),segFN=segments_1,generation=1}
[junit4] 2> 1492709 T8813 n:127.0.0.1:33817_ x:collection1 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 1492712 T8813 n:127.0.0.1:33817_ x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 1492713 T8813 n:127.0.0.1:33817_ x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 1492713 T8813 n:127.0.0.1:33817_ x:collection1 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 1492713 T8813 n:127.0.0.1:33817_ x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 1492713 T8813 n:127.0.0.1:33817_ x:collection1 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 1492713 T8813 n:127.0.0.1:33817_ x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 1492714 T8813 n:127.0.0.1:33817_ x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 1492714 T8813 n:127.0.0.1:33817_ x:collection1 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit
[...truncated too long message...]
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2>
[junit4] 2> 1509722 T8713 oejs.AbstractConnector.doStop Stopped ServerConnector@4dab548e{HTTP/1.1}{127.0.0.1:0}
[junit4] 2> 1509722 T8713 oejsh.ContextHandler.doStop Stopped o.e.j.s.ServletContextHandler@61c3db9c{/,null,UNAVAILABLE}
[junit4] 2> 1509725 T8713 c:control_collection s:shard1 x:collection1 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:40569 40569
[junit4] 2> 1509998 T8722 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:40569 40569
[junit4] 2> 1509999 T8722 oasc.ZkTestServer$ZKServerMain.runFromConfig WARN Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 7 /solr/aliases.json
[junit4] 2> 7 /solr/clusterstate.json
[junit4] 2> 5 /solr/configs/conf1
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 7 /solr/live_nodes
[junit4] 2> 5 /solr/overseer/collection-queue-work
[junit4] 2> 4 /solr/overseer/queue
[junit4] 2>
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestSolrCloudWithKerberos -Dtests.method=testKerberizedSolr -Dtests.seed=7F80B71B49A3C81 -Dtests.multiplier=2 -Dtests.slow=true -Dtests.locale=es_BO -Dtests.timezone=Canada/Newfoundland -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
[junit4] ERROR 31.8s J2 | TestSolrCloudWithKerberos.testKerberizedSolr <<<
[junit4] > Throwable #1: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:39003, http://127.0.0.1:51276, http://127.0.0.1:33817, http://127.0.0.1:40748, http://127.0.0.1:39669]
[junit4] > at __randomizedtesting.SeedInfo.seed([7F80B71B49A3C81:ACE1DDB6FEFBFCB6]:0)
[junit4] > at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:355)
[junit4] > at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1086)
[junit4] > at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:856)
[junit4] > at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:799)
[junit4] > at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:135)
[junit4] > at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:152)
[junit4] > at org.apache.solr.cloud.TestSolrCloudWithKerberos.testKerberizedSolr(TestSolrCloudWithKerberos.java:159)
[junit4] > at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:960)
[junit4] > at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:935)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] > Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:39669: Expected mime type application/octet-stream but got text/html. <html>
[junit4] > <head>
[junit4] > <meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/>
[junit4] > <title>Error 500 </title>
[junit4] > </head>
[junit4] > <body>
[junit4] > <h2>HTTP ERROR: 500</h2>
[junit4] > <p>Problem accessing /admin/collections. Reason:
[junit4] > <pre> org.apache.solr.common.SolrException: Error during request authentication, </pre></p>
[junit4] > <hr /><i><small>Powered by Jetty://</small></i>
[junit4] > </body>
[junit4] > </html>
[junit4] > at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:529)
[junit4] > at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:235)
[junit4] > at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:227)
[junit4] > at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:376)
[junit4] > at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:328)
[junit4] > ... 47 more
[junit4] 2> 1510012 T8713 c:control_collection s:shard1 x:collection1 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4] 2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001
[junit4] 2> 31774 T8712 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene50), sim=RandomSimilarityProvider(queryNorm=true,coord=crazy): {}, locale=es_BO, timezone=Canada/Newfoundland
[junit4] 2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.8.0_25 (64-bit)/cpus=4,threads=1,free=171080232,total=513802240
[junit4] 2> NOTE: All tests run in this JVM: [QueryParsingTest, SuggestComponentTest, OutputWriterTest, TestBinaryResponseWriter, HdfsUnloadDistributedZkTest, OutOfBoxZkACLAndCredentialsProvidersTest, DeleteShardTest, MultiThreadedOCPTest, TestStandardQParsers, SoftAutoCommitTest, XmlUpdateRequestHandlerTest, BasicDistributedZk2Test, DirectUpdateHandlerTest, SolrInfoMBeanTest, LoggingHandlerTest, TestManagedSchemaFieldResource, TestFieldCollectionResource, TestFastOutputStream, TestFieldTypeResource, RegexBoostProcessorTest, HdfsSyncSliceTest, TestPseudoReturnFields, DistributedQueryElevationComponentTest, TestSolrJ, TestSort, CircularListTest, SolrIndexSplitterTest, StatsComponentTest, TestSolrXml, TestSolrQueryParserDefaultOperatorResource, TestStressRecovery, TestQuerySenderNoQuery, DocValuesTest, TestSolr4Spatial2, SolrTestCaseJ4Test, FacetPivotSmallTest, SearchHandlerTest, TestManagedStopFilterFactory, TestShortCircuitedRequests, FileBasedSpellCheckerTest, TestMissingGroups, AsyncMigrateRouteKeyTest, DistribJoinFromCollectionTest, AutoCommitTest, CachingDirectoryFactoryTest, StressHdfsTest, QueryEqualityTest, TestSolrConfigHandler, RecoveryZkTest, MergeStrategyTest, TestRecovery, SolrCoreCheckLockOnStartupTest, TestDynamicFieldCollectionResource, TestExactSharedStatsCache, XsltUpdateRequestHandlerTest, TestMiniSolrCloudClusterSSL, TestCloudInspectUtil, AliasIntegrationTest, TestPartialUpdateDeduplication, TestCopyFieldCollectionResource, ResponseLogComponentTest, PluginInfoTest, DistributedIntervalFacetingTest, CustomCollectionTest, TestSearcherReuse, OverriddenZkACLAndCredentialsProvidersTest, DistanceUnitsTest, TestSolrQueryParser, VMParamsZkACLAndCredentialsProvidersTest, TestGroupingSearch, TestPerFieldSimilarity, HdfsRecoverLeaseTest, TestLeaderElectionZkExpiry, TestSystemIdResolver, TestMiniSolrCloudCluster, PolyFieldTest, SOLR749Test, TestZkChroot, CollectionsAPIDistributedZkTest, OverseerTest, ClusterStateUpdateTest, LeaderElectionTest, ShardRoutingCustomTest, TestFaceting, TestHashPartitioner, TestStressReorder, TestReload, TestStressVersions, SpellCheckComponentTest, BadIndexSchemaTest, TestBadConfig, CurrencyFieldXmlFileTest, AnalysisAfterCoreReloadTest, SolrRequestParserTest, TestTrie, StatelessScriptUpdateProcessorFactoryTest, DisMaxRequestHandlerTest, StandardRequestHandlerTest, TestQueryTypes, DocumentBuilderTest, TestCollationField, ReturnFieldsTest, MBeansHandlerTest, BinaryUpdateRequestHandlerTest, TestQuerySenderListener, TestSolrIndexConfig, TestMergePolicyConfig, SampleTest, TestDocSet, NumericFieldsTest, DirectSolrConnectionTest, NotRequiredUniqueKeyTest, SpellPossibilityIteratorTest, TestXIncludeConfig, TestSweetSpotSimilarityFactory, TestFastWriter, ScriptEngineTest, PrimUtilsTest, SpellingQueryConverterTest, DOMUtilTest, RAMDirectoryFactoryTest, ClusterStateTest, TestUtils, ZkNodePropsTest, TestRTGBase, TestCursorMarkWithoutUniqueKey, TestDistributedMissingSort, TestSimpleTrackingShardHandler, BaseCdcrDistributedZkTest, CdcrVersionReplicationTest, DistribCursorPagingTest, DistributedVersionInfoTest, OverseerStatusTest, ReplicationFactorTest, RollingRestartTest, SimpleCollectionCreateDeleteTest, SolrCloudExampleTest, TestRebalanceLeaders, TestSolrCloudWithKerberos]
[junit4] Completed [433/502] on J2 in 32.55s, 1 test, 1 error <<< FAILURES!
[...truncated 222 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/build.xml:474: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/build.xml:61: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/extra-targets.xml:39: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build.xml:229: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/common-build.xml:512: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/lucene/common-build.xml:1415: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/lucene/common-build.xml:973: There were test failures: 502 suites, 1984 tests, 1 error, 35 ignored (25 assumptions)
Total time: 50 minutes 23 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
No prior successful build to compare, so performing full copy of artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
[JENKINS] Lucene-Solr-Tests-trunk-Java8 - Build # 9 - Still Failing
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-trunk-Java8/9/
1 tests failed.
FAILED: org.apache.solr.cloud.TestSolrCloudWithKerberos.testKerberizedSolr
Error Message:
No live SolrServers available to handle this request:[http://127.0.0.1:43820, http://127.0.0.1:45701, http://127.0.0.1:45570, http://127.0.0.1:45919, http://127.0.0.1:41620]
Stack Trace:
org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:43820, http://127.0.0.1:45701, http://127.0.0.1:45570, http://127.0.0.1:45919, http://127.0.0.1:41620]
at __randomizedtesting.SeedInfo.seed([9840BAED8B010DDA:33596C2AC160CDED]:0)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:355)
at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1086)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:856)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:799)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:135)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:152)
at org.apache.solr.cloud.TestSolrCloudWithKerberos.testKerberizedSolr(TestSolrCloudWithKerberos.java:159)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:960)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:935)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:41620: Expected mime type application/octet-stream but got text/html. <html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/>
<title>Error 500 </title>
</head>
<body>
<h2>HTTP ERROR: 500</h2>
<p>Problem accessing /admin/collections. Reason:
<pre> org.apache.solr.common.SolrException: Error during request authentication, </pre></p>
<hr /><i><small>Powered by Jetty://</small></i>
</body>
</html>
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:529)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:235)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:227)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:376)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:328)
... 47 more
Build Log:
[...truncated 10484 lines...]
[junit4] Suite: org.apache.solr.cloud.TestSolrCloudWithKerberos
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 9840BAED8B010DDA-001/init-core-data-001
[junit4] 2> 1424189 T8006 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
[junit4] 2> 1427307 T8006 oadsc.DefaultDirectoryService.showSecurityWarnings WARN You didn't change the admin password of directory service instance 'DefaultKrbServer'. Please update the admin password as soon as possible to prevent a possible security breach.
[junit4] 2> 1427585 T8006 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 2> 1427586 T8014 oasc.ZkTestServer$2$1.setClientPort client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 1427586 T8014 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 1427686 T8006 oasc.ZkTestServer.run start zk server on port:57183
[junit4] 2> 1427687 T8006 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
[junit4] 2> 1427688 T8006 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1427688 T8019 oaz.ClientCnxn$SendThread.startConnect WARN SASL configuration failed: javax.security.auth.login.LoginException: No JAAS configuration section named 'Client' was found in specified JAAS configuration file: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 9840BAED8B010DDA-001/tempDir-001/minikdc/jaas-client.conf'. Will continue connection to Zookeeper server without SASL authentication, if Zookeeper server allows it.
[junit4] 2> 1427689 T8021 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3e0b0c29 name:ZooKeeperConnection Watcher:127.0.0.1:57183 got event WatchedEvent state:AuthFailed type:None path:null path:null type:None
[junit4] 2> 1427689 T8021 oascc.ConnectionManager.process WARN zkClient received AuthFailed
[junit4] 2> 1427691 T8021 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3e0b0c29 name:ZooKeeperConnection Watcher:127.0.0.1:57183 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1427691 T8006 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1427691 T8006 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
[junit4] 2> 1427691 T8006 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 1427700 T8006 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
[junit4] 2> 1427703 T8022 oaz.ClientCnxn$SendThread.startConnect WARN SASL configuration failed: javax.security.auth.login.LoginException: No JAAS configuration section named 'Client' was found in specified JAAS configuration file: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 9840BAED8B010DDA-001/tempDir-001/minikdc/jaas-client.conf'. Will continue connection to Zookeeper server without SASL authentication, if Zookeeper server allows it.
[junit4] 2> 1427707 T8006 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1427707 T8024 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@14207114 name:ZooKeeperConnection Watcher:127.0.0.1:57183/solr got event WatchedEvent state:AuthFailed type:None path:null path:null type:None
[junit4] 2> 1427708 T8024 oascc.ConnectionManager.process WARN zkClient received AuthFailed
[junit4] 2> 1427707 T8025 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@14207114 name:ZooKeeperConnection Watcher:127.0.0.1:57183/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1427708 T8006 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1427708 T8006 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
[junit4] 2> 1427709 T8006 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4] 2> 1427710 T8006 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4] 2> 1427712 T8006 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4] 2> 1427713 T8006 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4] 2> 1427715 T8006 oasc.AbstractZkTestCase.putConfig put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 1427715 T8006 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4] 2> 1427717 T8006 oasc.AbstractZkTestCase.putConfig put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4] 2> 1427718 T8006 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4] 2> 1427719 T8006 oasc.AbstractZkTestCase.putConfig put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 1427720 T8006 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 1427721 T8006 oasc.AbstractZkTestCase.putConfig put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 1427722 T8006 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4] 2> 1427723 T8006 oasc.AbstractZkTestCase.putConfig put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 1427724 T8006 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4] 2> 1427725 T8006 oasc.AbstractZkTestCase.putConfig put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 1427725 T8006 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4] 2> 1427727 T8006 oasc.AbstractZkTestCase.putConfig put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 1427727 T8006 oascc.SolrZkClient.makePath makePath: /configs/conf1/enumsConfig.xml
[junit4] 2> 1427729 T8006 oasc.AbstractZkTestCase.putConfig put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 1427729 T8006 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4] 2> 1427731 T8006 oasc.AbstractZkTestCase.putConfig put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 1427732 T8006 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 1427733 T8006 oasc.AbstractZkTestCase.putConfig put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 1427733 T8006 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4] 2> 1427735 T8006 oasc.AbstractZkTestCase.putConfig put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 1427735 T8006 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4] 2> 1427737 T8006 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
[junit4] 2> 1427739 T8026 oaz.ClientCnxn$SendThread.startConnect WARN SASL configuration failed: javax.security.auth.login.LoginException: No JAAS configuration section named 'Client' was found in specified JAAS configuration file: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 9840BAED8B010DDA-001/tempDir-001/minikdc/jaas-client.conf'. Will continue connection to Zookeeper server without SASL authentication, if Zookeeper server allows it.
[junit4] 2> 1427740 T8006 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1427743 T8028 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@55bce6c1 name:ZooKeeperConnection Watcher:127.0.0.1:57183/solr got event WatchedEvent state:AuthFailed type:None path:null path:null type:None
[junit4] 2> 1427743 T8028 oascc.ConnectionManager.process WARN zkClient received AuthFailed
[junit4] 2> 1427743 T8029 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@55bce6c1 name:ZooKeeperConnection Watcher:127.0.0.1:57183/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1427743 T8006 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1427744 T8006 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
[junit4] 2> 1428085 T8006 oas.SolrTestCaseJ4.writeCoreProperties Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 9840BAED8B010DDA-001/control-001/cores/collection1
[junit4] 2> 1428087 T8006 oejs.Server.doStart jetty-9.2.10.v20150310
[junit4] 2> 1428089 T8006 oejsh.ContextHandler.doStart Started o.e.j.s.ServletContextHandler@193b859c{/,null,AVAILABLE}
[junit4] 2> 1428089 T8006 oejs.AbstractConnector.doStart Started ServerConnector@3d1ae0e4{HTTP/1.1}{127.0.0.1:41620}
[junit4] 2> 1428090 T8006 oejs.Server.doStart Started @1430179ms
[junit4] 2> 1428090 T8006 oascse.JettySolrRunner$1.lifeCycleStarted Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 9840BAED8B010DDA-001/tempDir-002/control/data, hostContext=/, hostPort=41620, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 9840BAED8B010DDA-001/control-001/cores}
[junit4] 2> 1428090 T8006 oass.SolrDispatchFilter.init SolrDispatchFilter.init()sun.misc.Launcher$AppClassLoader@c387f44
[junit4] 2> 1428091 T8006 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 9840BAED8B010DDA-001/control-001/'
[junit4] 2> 1428106 T8006 oasc.SolrXmlConfig.fromFile Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 9840BAED8B010DDA-001/control-001/solr.xml
[junit4] 2> 1428113 T8006 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 9840BAED8B010DDA-001/control-001/cores
[junit4] 2> 1428113 T8006 oasc.CoreContainer.<init> New CoreContainer 992021330
[junit4] 2> 1428113 T8006 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 9840BAED8B010DDA-001/control-001/]
[junit4] 2> 1428114 T8006 oasc.CoreContainer.load loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 9840BAED8B010DDA-001/control-001/lib
[junit4] 2> 1428114 T8006 oasc.SolrResourceLoader.addToClassLoader WARN Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 9840BAED8B010DDA-001/control-001/lib).
[junit4] 2> 1428121 T8006 oashc.HttpShardHandlerFactory.init created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
[junit4] 2> 1428123 T8006 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 1428123 T8006 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 1428123 T8006 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 1428124 T8006 oasc.CoreContainer.load Node Name: 127.0.0.1
[junit4] 2> 1428124 T8006 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:57183/solr
[junit4] 2> 1428124 T8006 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 1428125 T8006 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
[junit4] 2> 1428125 T8006 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1428125 T8041 oaz.ClientCnxn$SendThread.startConnect WARN SASL configuration failed: javax.security.auth.login.LoginException: No JAAS configuration section named 'Client' was found in specified JAAS configuration file: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 9840BAED8B010DDA-001/tempDir-001/minikdc/jaas-client.conf'. Will continue connection to Zookeeper server without SASL authentication, if Zookeeper server allows it.
[junit4] 2> 1428126 T8043 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@32746ce8 name:ZooKeeperConnection Watcher:127.0.0.1:57183 got event WatchedEvent state:AuthFailed type:None path:null path:null type:None
[junit4] 2> 1428127 T8044 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@32746ce8 name:ZooKeeperConnection Watcher:127.0.0.1:57183 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1428127 T8006 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1428128 T8006 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
[junit4] 2> 1428127 T8043 oascc.ConnectionManager.process WARN zkClient received AuthFailed
[junit4] 2> 1428129 T8045 n:127.0.0.1:41620_ oaz.ClientCnxn$SendThread.startConnect WARN SASL configuration failed: javax.security.auth.login.LoginException: No JAAS configuration section named 'Client' was found in specified JAAS configuration file: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 9840BAED8B010DDA-001/tempDir-001/minikdc/jaas-client.conf'. Will continue connection to Zookeeper server without SASL authentication, if Zookeeper server allows it.
[junit4] 2> 1428129 T8006 n:127.0.0.1:41620_ oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1428130 T8047 n:127.0.0.1:41620_ oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6c41d756 name:ZooKeeperConnection Watcher:127.0.0.1:57183/solr got event WatchedEvent state:AuthFailed type:None path:null path:null type:None
[junit4] 2> 1428130 T8047 n:127.0.0.1:41620_ oascc.ConnectionManager.process WARN zkClient received AuthFailed
[junit4] 2> 1428130 T8048 n:127.0.0.1:41620_ oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6c41d756 name:ZooKeeperConnection Watcher:127.0.0.1:57183/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1428131 T8006 n:127.0.0.1:41620_ oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1428132 T8006 n:127.0.0.1:41620_ oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 1428136 T8006 n:127.0.0.1:41620_ oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 1428138 T8006 n:127.0.0.1:41620_ oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
[junit4] 2> 1428140 T8006 n:127.0.0.1:41620_ oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
[junit4] 2> 1428142 T8006 n:127.0.0.1:41620_ oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
[junit4] 2> 1428144 T8006 n:127.0.0.1:41620_ oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 1428145 T8006 n:127.0.0.1:41620_ oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 1428147 T8006 n:127.0.0.1:41620_ oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 1428148 T8006 n:127.0.0.1:41620_ oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:41620_
[junit4] 2> 1428148 T8006 n:127.0.0.1:41620_ oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:41620_
[junit4] 2> 1428150 T8006 n:127.0.0.1:41620_ oascc.SolrZkClient.makePath makePath: /overseer_elect
[junit4] 2> 1428151 T8006 n:127.0.0.1:41620_ oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 1428152 T8006 n:127.0.0.1:41620_ oasc.Overseer.close Overseer (id=null) closing
[junit4] 2> 1428154 T8006 n:127.0.0.1:41620_ oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:41620_
[junit4] 2> 1428154 T8006 n:127.0.0.1:41620_ oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 1428155 T8006 n:127.0.0.1:41620_ oasc.Overseer.start Overseer (id=93882404124884996-127.0.0.1:41620_-n_0000000000) starting
[junit4] 2> 1428157 T8006 n:127.0.0.1:41620_ oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 1428162 T8006 n:127.0.0.1:41620_ oasc.OverseerAutoReplicaFailoverThread.<init> Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
[junit4] 2> 1428163 T8006 n:127.0.0.1:41620_ oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1428163 T8050 n:127.0.0.1:41620_ oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 1428163 T8049 n:127.0.0.1:41620_ oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 1428166 T8006 n:127.0.0.1:41620_ oasc.CoreContainer.initializeAuthenticationPlugin Authentication plugin class obtained from ZK: org.apache.solr.security.KerberosPlugin
[junit4] 2> 1428173 T8006 n:127.0.0.1:41620_ oass.KerberosPlugin.init Params: {token.valid=30, kerberos.principal=HTTP/127.0.0.1, cookie.domain=127.0.0.1, kerberos.keytab=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 9840BAED8B010DDA-001/tempDir-001/minikdc/keytabs, cookie.path=/, type=kerberos, kerberos.name.rules=DEFAULT}
[junit4] 2> 1428185 T8006 n:127.0.0.1:41620_ oasc.CoreContainer.initializeAuthenticationPlugin Reconfiguring the shard handler factory and update shard handler.
[junit4] 2> 1428185 T8006 n:127.0.0.1:41620_ oashc.HttpShardHandlerFactory.reconfigureHttpClient Reconfiguring the default client with: org.apache.solr.client.solrj.impl.Krb5HttpClientConfigurer@5ec6ebfa
[junit4] 2> 1428185 T8006 n:127.0.0.1:41620_ oascsi.Krb5HttpClientConfigurer.configure Setting up SPNego auth with config: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 9840BAED8B010DDA-001/tempDir-001/minikdc/jaas-client.conf
[junit4] 2> 1428188 T8006 n:127.0.0.1:41620_ oasu.UpdateShardHandler.reconfigureHttpClient Reconfiguring the default client with: org.apache.solr.client.solrj.impl.Krb5HttpClientConfigurer@5ec6ebfa
[junit4] 2> 1428189 T8006 n:127.0.0.1:41620_ oascsi.Krb5HttpClientConfigurer.configure Setting up SPNego auth with config: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 9840BAED8B010DDA-001/tempDir-001/minikdc/jaas-client.conf
[junit4] 2> 1428190 T8006 n:127.0.0.1:41620_ oasc.CorePropertiesLocator.discover Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 9840BAED8B010DDA-001/control-001/cores
[junit4] 2> 1428191 T8006 n:127.0.0.1:41620_ oasc.CoreDescriptor.<init> CORE DESCRIPTOR: {name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, loadOnStartup=true, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 9840BAED8B010DDA-001/control-001/cores/collection1, collection=control_collection, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 9840BAED8B010DDA-001/control-001/cores/collection1/, coreNodeName=, dataDir=data/, shard=}
[junit4] 2> 1428191 T8006 n:127.0.0.1:41620_ oasc.CorePropertiesLocator.discoverUnder Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 9840BAED8B010DDA-001/control-001/cores/collection1/
[junit4] 2> 1428192 T8006 n:127.0.0.1:41620_ oasc.CorePropertiesLocator.discover Found 1 core definitions
[junit4] 2> 1428192 T8053 n:127.0.0.1:41620_ c:control_collection x:collection1 oasc.ZkController.publish publishing core=collection1 state=down collection=control_collection
[junit4] 2> 1428193 T8053 n:127.0.0.1:41620_ c:control_collection x:collection1 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 1428193 T8048 n:127.0.0.1:41620_ oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
[junit4] 2> 1428193 T8053 n:127.0.0.1:41620_ oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 1428194 T8049 n:127.0.0.1:41620_ oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"http://127.0.0.1:41620",
[junit4] 2> "node_name":"127.0.0.1:41620_",
[junit4] 2> "numShards":"1",
[junit4] 2> "state":"down",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"control_collection",
[junit4] 2> "operation":"state"} current state version: 0
[junit4] 2> 1428195 T8049 n:127.0.0.1:41620_ oasco.ReplicaMutator.updateState Update state numShards=1 message={
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"http://127.0.0.1:41620",
[junit4] 2> "node_name":"127.0.0.1:41620_",
[junit4] 2> "numShards":"1",
[junit4] 2> "state":"down",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"control_collection",
[junit4] 2> "operation":"state"}
[junit4] 2> 1428195 T8049 n:127.0.0.1:41620_ oasco.ClusterStateMutator.createCollection building a new cName: control_collection
[junit4] 2> 1428195 T8049 n:127.0.0.1:41620_ oasco.ReplicaMutator.updateState Assigning new node to shard shard=shard1
[junit4] 2> 1428196 T8048 n:127.0.0.1:41620_ oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1429194 T8053 n:127.0.0.1:41620_ oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 1429194 T8053 n:127.0.0.1:41620_ oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4] 2> 1429195 T8053 n:127.0.0.1:41620_ oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 1429195 T8053 n:127.0.0.1:41620_ oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
[junit4] 2> 1429196 T8053 n:127.0.0.1:41620_ oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 1429196 T8053 n:127.0.0.1:41620_ oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 9840BAED8B010DDA-001/control-001/cores/collection1/'
[junit4] 2> 1429208 T8053 n:127.0.0.1:41620_ oasc.Config.<init> loaded config solrconfig.xml with version 0
[junit4] 2> 1429222 T8053 n:127.0.0.1:41620_ oasc.SolrConfig.refreshRequestParams current version of requestparams : -1
[junit4] 2> 1429227 T8053 n:127.0.0.1:41620_ oasc.SolrConfig.<init> Using Lucene MatchVersion: 6.0.0
[junit4] 2> 1429236 T8053 n:127.0.0.1:41620_ oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 1429238 T8053 n:127.0.0.1:41620_ oass.IndexSchema.readSchema Reading Solr Schema from /configs/conf1/schema.xml
[junit4] 2> 1429242 T8053 n:127.0.0.1:41620_ oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 1429364 T8053 n:127.0.0.1:41620_ oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 1429371 T8053 n:127.0.0.1:41620_ oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 1429372 T8053 n:127.0.0.1:41620_ oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 1429379 T8053 n:127.0.0.1:41620_ oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 1429381 T8053 n:127.0.0.1:41620_ oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 1429383 T8053 n:127.0.0.1:41620_ oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 1429384 T8053 n:127.0.0.1:41620_ oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 1429394 T8053 n:127.0.0.1:41620_ oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 1429395 T8053 n:127.0.0.1:41620_ oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 1429396 T8053 n:127.0.0.1:41620_ oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 1429396 T8053 n:127.0.0.1:41620_ oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 1429396 T8053 n:127.0.0.1:41620_ oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection control_collection
[junit4] 2> 1429397 T8053 n:127.0.0.1:41620_ x:collection1 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 1429397 T8053 n:127.0.0.1:41620_ x:collection1 oasc.SolrCore.<init> [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 9840BAED8B010DDA-001/control-001/cores/collection1/], dataDir=[null]
[junit4] 2> 1429397 T8053 n:127.0.0.1:41620_ x:collection1 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@513effe
[junit4] 2> 1429398 T8053 n:127.0.0.1:41620_ x:collection1 oasc.CachingDirectoryFactory.get return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 9840BAED8B010DDA-001/control-001/cores/collection1/data
[junit4] 2> 1429398 T8053 n:127.0.0.1:41620_ x:collection1 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 9840BAED8B010DDA-001/control-001/cores/collection1/data/index/
[junit4] 2> 1429398 T8053 n:127.0.0.1:41620_ x:collection1 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 9840BAED8B010DDA-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
[junit4] 2> 1429398 T8053 n:127.0.0.1:41620_ x:collection1 oasc.CachingDirectoryFactory.get return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 9840BAED8B010DDA-001/control-001/cores/collection1/data/index
[junit4] 2> 1429399 T8053 n:127.0.0.1:41620_ x:collection1 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=10, maxMergeAtOnceExplicit=40, maxMergedSegmentMB=46.45703125, floorSegmentMB=0.359375, forceMergeDeletesPctAllowed=4.1970158864613, segmentsPerTier=34.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
[junit4] 2> 1429399 T8053 n:127.0.0.1:41620_ x:collection1 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@4c2fe812 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@36d9f691),segFN=segments_1,generation=1}
[junit4] 2> 1429400 T8053 n:127.0.0.1:41620_ x:collection1 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 1429403 T8053 n:127.0.0.1:41620_ x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 1429403 T8053 n:127.0.0.1:41620_ x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 1429403 T8053 n:127.0.0.1:41620_ x:collection1 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 1429404 T8053 n:127.0.0.1:41620_ x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 1429404 T8053 n:127.0.0.1:41620_ x:collection1 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 1429404 T8053 n:127.0.0.1:41620_ x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 1429404 T8053 n:127.0.0.1:41620_ x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 1429404 T8053 n:127.0.0.1:41620_ x:collection1 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 1429405 T8053 n:127.0.0.1:41620_ x:collection1 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 1429407 T8053 n:127.0.0.1:41620_ x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 1429408 T8053 n:127.0.0.1:41620_ x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 1429409 T8053 n:127.0.0.1:41620_ x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 1429410 T8053 n:127.0.0.1:41620_ x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 1429412 T8053 n:127.0.0.1:41620_ x:collection1 oasc.RequestHandlers.initHandlersFromConfig Registered paths: /admin/mbeans,standard,/update/csv,/update/json/docs,/admin/luke,/admin/segments,/get,/admin/system,/replication,/admin/properties,/config,/schema,/admin/plugins,/admin/logging,/update/json,/admin/threads,/admin/ping,/update,/admin/file
[junit4] 2> 1429413 T8053 n:127.0.0.1:41620_ x:collection1 oasc.SolrCore.initStatsCache Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
[junit4] 2> 1429414 T8053 n:127.0.0.1:41620_ x:collection1 oasu.UpdateHandler.<init> Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 1429414 T8053 n:127.0.0.1:41620_ x:collection1 oasu.UpdateLog.init Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 1429415 T8053 n:127.0.0.1:41620_ x:collection1 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 1429415 T8053 n:127.0.0.1:41620_ x:collection1 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 1429416 T8053 n:127.0.0.1:41620_ x:collection1 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=27, maxMergeAtOnceExplicit=46, maxMergedSegmentMB=61.61328125, floorSegmentMB=1.8525390625, forceMergeDeletesPctAllowed=10.319260276732745, segmentsPerTier=46.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> 1429416 T8053 n:127.0.0.1:41620_ x:collection1 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@4c2fe812 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@36d9f691),segFN=segments_1,generation=1}
[junit4] 2> 1429417 T8053 n:127.0.0.1:41620_ x:collection1 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 1429417 T8053 n:127.0.0.1:41620_ x:collection1 oass.SolrIndexSearcher.<init> Opening Searcher@91b28d[collection1] main
[junit4] 2> 1429417 T8053 n:127.0.0.1:41620_ x:collection1 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
[junit4] 2> 1429418 T8053 n:127.0.0.1:41620_ x:collection1 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 1429418 T8053 n:127.0.0.1:41620_ x:collection1 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
[junit4] 2> 1429419 T8053 n:127.0.0.1:41620_ x:collection1 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 1429419 T8053 n:127.0.0.1:41620_ x:collection1 oasr.RestManager.init Initializing RestManager with initArgs: {}
[junit4] 2> 1429419 T8053 n:127.0.0.1:41620_ x:collection1 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1429420 T8053 n:127.0.0.1:41620_ x:collection1 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json
[junit4] 2> 1429420 T8053 n:127.0.0.1:41620_ x:collection1 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1429420 T8053 n:127.0.0.1:41620_ x:collection1 oasr.RestManager.init Initializing 0 registered ManagedResources
[junit4] 2> 1429420 T8053 n:127.0.0.1:41620_ x:collection1 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4] 2> 1429421 T8053 n:127.0.0.1:41620_ x:collection1 oasc.ZkController.getConfDirListeners watch zkdir /configs/conf1
[junit4] 2> 1429421 T8054 n:127.0.0.1:41620_ x:collection1 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@91b28d[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1429421 T8053 n:127.0.0.1:41620_ x:collection1 oasu.UpdateLog.onFirstSearcher On first searcher opened, looking up max value of version field
[junit4] 2> 1429422 T8053 n:127.0.0.1:41620_ x:collection1 oasu.VersionInfo.getMaxVersionFromIndex Refreshing highest value of _version_ for 65536 version buckets from index
[junit4] 2> 1429422 T8053 n:127.0.0.1:41620_ x:collection1 oasu.VersionInfo.getMaxVersionFromIndex No terms found for _version_, cannot seed version bucket highest value from index
[junit4] 2> 1429422 T8053 n:127.0.0.1:41620_ x:collection1 oasu.UpdateLog.seedBucketsWithHighestVersion Could not find max version in index or recent updates, using new clock 1502118467918102528
[junit4] 2> 1429424 T8053 n:127.0.0.1:41620_ x:collection1 oasu.UpdateLog.seedBucketsWithHighestVersion Took 2 ms to seed version buckets with highest version 1502118467918102528
[junit4] 2> 1429424 T8053 n:127.0.0.1:41620_ x:collection1 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 1429425 T8057 n:127.0.0.1:41620_ c:control_collection s:shard1 x:collection1 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:41620 collection:control_collection shard:shard1
[junit4] 2> 1429426 T8006 n:127.0.0.1:41620_ oascsi.Krb5HttpClientConfigurer.configure Setting up SPNego auth with config: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 9840BAED8B010DDA-001/tempDir-001/minikdc/jaas-client.conf
[junit4] 2> 1429426 T8006 n:127.0.0.1:41620_ oass.SolrDispatchFilter.init user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2
[junit4] 2> 1429426 T8006 n:127.0.0.1:41620_ oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 1429427 T8006 oascsi.Krb5HttpClientConfigurer.configure Setting up SPNego auth with config: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 9840BAED8B010DDA-001/tempDir-001/minikdc/jaas-client.conf
[junit4] 2> 1429428 T8006 oascsi.Krb5HttpClientConfigurer.configure Setting up SPNego auth with config: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 9840BAED8B010DDA-001/tempDir-001/minikdc/jaas-client.conf
[junit4] 2> 1429428 T8006 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
[junit4] 2> 1429429 T8058 oaz.ClientCnxn$SendThread.startConnect WARN SASL configuration failed: javax.security.auth.login.LoginException: No JAAS configuration section named 'Client' was found in specified JAAS configuration file: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 9840BAED8B010DDA-001/tempDir-001/minikdc/jaas-client.conf'. Will continue connection to Zookeeper server without SASL authentication, if Zookeeper server allows it.
[junit4] 2> 1429430 T8057 n:127.0.0.1:41620_ c:control_collection s:shard1 x:collection1 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4] 2> 1429430 T8006 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1429433 T8060 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2f1e0b6 name:ZooKeeperConnection Watcher:127.0.0.1:57183/solr got event WatchedEvent state:AuthFailed type:None path:null path:null type:None
[junit4] 2> 1429433 T8060 oascc.ConnectionManager.process WARN zkClient received AuthFailed
[junit4] 2> 1429434 T8057 n:127.0.0.1:41620_ c:control_collection s:shard1 x:collection1 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 1429435 T8048 n:127.0.0.1:41620_ oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
[junit4] 2> 1429436 T8049 n:127.0.0.1:41620_ oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"leader",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"control_collection"} current state version: 1
[junit4] 2> 1429436 T8061 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2f1e0b6 name:ZooKeeperConnection Watcher:127.0.0.1:57183/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1429437 T8006 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1429438 T8006 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
[junit4] 2> 1429438 T8006 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1429438 T8057 n:127.0.0.1:41620_ c:control_collection s:shard1 x:collection1 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 1429439 T8057 n:127.0.0.1:41620_ c:control_collection s:shard1 x:collection1 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C1286 name=collection1 org.apache.solr.core.SolrCore@371c3455 url=http://127.0.0.1:41620/collection1 node=127.0.0.1:41620_ C1286_STATE=coll:control_collection core:collection1 props:{core=collection1, base_url=http://127.0.0.1:41620, node_name=127.0.0.1:41620_, state=down}
[junit4] 2> 1429439 T8057 n:127.0.0.1:41620_ c:control_collection s:shard1 x:collection1 C1286 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:41620/collection1/
[junit4] 2> 1429440 T8006 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 1429440 T8006 oasc.AbstractFullDistribZkTestBase.createJettys Creating collection1 with stateFormat=2
[junit4] 2> 1429441 T8006 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
[junit4] 2> 1429441 T8057 n:127.0.0.1:41620_ c:control_collection s:shard1 x:collection1 C1286 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 1429441 T8062 oaz.ClientCnxn$SendThread.startConnect WARN SASL configuration failed: javax.security.auth.login.LoginException: No JAAS configuration section named 'Client' was found in specified JAAS configuration file: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 9840BAED8B010DDA-001/tempDir-001/minikdc/jaas-client.conf'. Will continue connection to Zookeeper server without SASL authentication, if Zookeeper server allows it.
[junit4] 2> 1429441 T8006 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1429441 T8057 n:127.0.0.1:41620_ c:control_collection s:shard1 x:collection1 C1286 oasc.SyncStrategy.syncToMe http://127.0.0.1:41620/collection1/ has no replicas
[junit4] 2> 1429442 T8057 n:127.0.0.1:41620_ c:control_collection s:shard1 x:collection1 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:41620/collection1/ shard1
[junit4] 2> 1429443 T8057 n:127.0.0.1:41620_ c:control_collection s:shard1 x:collection1 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4] 2> 1429445 T8048 n:127.0.0.1:41620_ oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
[junit4] 2> 1429446 T8049 n:127.0.0.1:41620_ oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"leader",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"control_collection",
[junit4] 2> "base_url":"http://127.0.0.1:41620",
[junit4] 2> "core":"collection1",
[junit4] 2> "state":"active"} current state version: 1
[junit4] 2> 1429448 T8065 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@29f79975 name:ZooKeeperConnection Watcher:127.0.0.1:57183/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1429448 T8006 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1429448 T8006 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
[junit4] 2> 1429450 T8048 n:127.0.0.1:41620_ oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
[junit4] 2> 1429450 T8049 n:127.0.0.1:41620_ oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"create",
[junit4] 2> "name":"collection1",
[junit4] 2> "numShards":"2",
[junit4] 2> "stateFormat":"2"} current state version: 1
[junit4] 2> 1429450 T8049 n:127.0.0.1:41620_ oasco.ClusterStateMutator.createCollection building a new cName: collection1
[junit4] 2> 1429451 T8061 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1429451 T8048 n:127.0.0.1:41620_ oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1429452 T8064 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@29f79975 name:ZooKeeperConnection Watcher:127.0.0.1:57183/solr got event WatchedEvent state:AuthFailed type:None path:null path:null type:None
[junit4] 2> 1429452 T8064 oascc.ConnectionManager.process Client->ZooKeeper status change trigger but we are already closed
[junit4] 2> 1429498 T8057 n:127.0.0.1:41620_ c:control_collection s:shard1 x:collection1 oasc.ZkController.register We are http://127.0.0.1:41620/collection1/ and leader is http://127.0.0.1:41620/collection1/
[junit4] 2> 1429498 T8057 n:127.0.0.1:41620_ c:control_collection s:shard1 x:collection1 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:41620
[junit4] 2> 1429498 T8057 n:127.0.0.1:41620_ c:control_collection s:shard1 x:collection1 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 1429499 T8057 n:127.0.0.1:41620_ c:control_collection s:shard1 x:collection1 oasc.ZkController.publish publishing core=collection1 state=active collection=control_collection
[junit4] 2> 1429499 T8057 n:127.0.0.1:41620_ c:control_collection s:shard1 x:collection1 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 1429500 T8048 n:127.0.0.1:41620_ oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
[junit4] 2> 1429501 T8049 n:127.0.0.1:41620_ oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
[junit4] 2> "core":"collection1",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"http://127.0.0.1:41620",
[junit4] 2> "node_name":"127.0.0.1:41620_",
[junit4] 2> "numShards":"2",
[junit4] 2> "state":"active",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"control_collection",
[junit4] 2> "operation":"state"} current state version: 2
[junit4] 2> 1429501 T8049 n:127.0.0.1:41620_ oasco.ReplicaMutator.updateState Update state numShards=2 message={
[junit4] 2> "core":"collection1",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"http://127.0.0.1:41620",
[junit4] 2> "node_name":"127.0.0.1:41620_",
[junit4] 2> "numShards":"2",
[junit4] 2> "state":"active",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"control_collection",
[junit4] 2> "operation":"state"}
[junit4] 2> 1429502 T8049 n:127.0.0.1:41620_ oasco.ZkStateWriter.writePendingUpdates going to create_collection /collections/collection1/state.json
[junit4] 2> 1429503 T8061 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1429504 T8048 n:127.0.0.1:41620_ oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1429608 T8061 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1429608 T8048 n:127.0.0.1:41620_ oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1429801 T8006 oas.SolrTestCaseJ4.writeCoreProperties Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 9840BAED8B010DDA-001/shard-1-001/cores/collection1
[junit4] 2> 1429802 T8006 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 9840BAED8B010DDA-001/shard-1-001
[junit4] 2> 1429803 T8006 oejs.Server.doStart jetty-9.2.10.v20150310
[junit4] 2> 1429805 T8006 oejsh.ContextHandler.doStart Started o.e.j.s.ServletContextHandler@6c58ada5{/,null,AVAILABLE}
[junit4] 2> 1429805 T8006 oejs.AbstractConnector.doStart Started ServerConnector@29768aa3{HTTP/1.1}{127.0.0.1:43820}
[junit4] 2> 1429805 T8006 oejs.Server.doStart Started @1431895ms
[junit4] 2> 1429806 T8006 oascse.JettySolrRunner$1.lifeCycleStarted Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 9840BAED8B010DDA-001/tempDir-002/jetty1, solrconfig=solrconfig.xml, hostContext=/, hostPort=43820, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 9840BAED8B010DDA-001/shard-1-001/cores}
[junit4] 2> 1429806 T8006 oascsi.Krb5HttpClientConfigurer.configure Setting up SPNego auth with config: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 9840BAED8B010DDA-001/tempDir-001/minikdc/jaas-client.conf
[junit4] 2> 1429807 T8006 oass.SolrDispatchFilter.init SolrDispatchFilter.init()sun.misc.Launcher$AppClassLoader@c387f44
[junit4] 2> 1429807 T8006 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 9840BAED8B010DDA-001/shard-1-001/'
[junit4] 2> 1429821 T8006 oasc.SolrXmlConfig.fromFile Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 9840BAED8B010DDA-001/shard-1-001/solr.xml
[junit4] 2> 1429827 T8006 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 9840BAED8B010DDA-001/shard-1-001/cores
[junit4] 2> 1429827 T8006 oasc.CoreContainer.<init> New CoreContainer 1300125204
[junit4] 2> 1429828 T8006 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 9840BAED8B010DDA-001/shard-1-001/]
[junit4] 2> 1429828 T8006 oasc.CoreContainer.load loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 9840BAED8B010DDA-001/shard-1-001/lib
[junit4] 2> 1429828 T8006 oasc.SolrResourceLoader.addToClassLoader WARN Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 9840BAED8B010DDA-001/shard-1-001/lib).
[junit4] 2> 1429835 T8006 oashc.HttpShardHandlerFactory.init created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
[junit4] 2> 1429837 T8006 oascsi.Krb5HttpClientConfigurer.configure Setting up SPNego auth with config: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 9840BAED8B010DDA-001/tempDir-001/minikdc/jaas-client.conf
[junit4] 2> 1429837 T8006 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 1429838 T8006 oascsi.Krb5HttpClientConfigurer.configure Setting up SPNego auth with config: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 9840BAED8B010DDA-001/tempDir-001/minikdc/jaas-client.conf
[junit4] 2> 1429838 T8006 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 1429838 T8006 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 1429839 T8006 oasc.CoreContainer.load Node Name: 127.0.0.1
[junit4] 2> 1429839 T8006 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:57183/solr
[junit4] 2> 1429839 T8006 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 1429840 T8006 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
[junit4] 2> 1429840 T8006 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1429840 T8077 oaz.ClientCnxn$SendThread.startConnect WARN SASL configuration failed: javax.security.auth.login.LoginException: No JAAS configuration section named 'Client' was found in specified JAAS configuration file: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 9840BAED8B010DDA-001/tempDir-001/minikdc/jaas-client.conf'. Will continue connection to Zookeeper server without SASL authentication, if Zookeeper server allows it.
[junit4] 2> 1429841 T8079 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7eb5a740 name:ZooKeeperConnection Watcher:127.0.0.1:57183 got event WatchedEvent state:AuthFailed type:None path:null path:null type:None
[junit4] 2> 1429841 T8079 oascc.ConnectionManager.process WARN zkClient received AuthFailed
[junit4] 2> 1429842 T8079 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7eb5a740 name:ZooKeeperConnection Watcher:127.0.0.1:57183 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1429842 T8006 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1429842 T8006 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
[junit4] 2> 1429844 T8006 n:127.0.0.1:43820_ oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1429844 T8080 n:127.0.0.1:43820_ oaz.ClientCnxn$SendThread.startConnect WARN SASL configuration failed: javax.security.auth.login.LoginException: No JAAS configuration section named 'Client' was found in specified JAAS configuration file: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 9840BAED8B010DDA-001/tempDir-001/minikdc/jaas-client.conf'. Will continue connection to Zookeeper server without SASL authentication, if Zookeeper server allows it.
[junit4] 2> 1429847 T8082 n:127.0.0.1:43820_ oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2b6e04ab name:ZooKeeperConnection Watcher:127.0.0.1:57183/solr got event WatchedEvent state:AuthFailed type:None path:null path:null type:None
[junit4] 2> 1429847 T8082 n:127.0.0.1:43820_ oascc.ConnectionManager.process WARN zkClient received AuthFailed
[junit4] 2> 1429847 T8083 n:127.0.0.1:43820_ oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2b6e04ab name:ZooKeeperConnection Watcher:127.0.0.1:57183/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1429847 T8006 n:127.0.0.1:43820_ oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1429851 T8006 n:127.0.0.1:43820_ oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1430856 T8006 n:127.0.0.1:43820_ oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:43820_
[junit4] 2> 1430857 T8006 n:127.0.0.1:43820_ oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:43820_
[junit4] 2> 1430860 T8006 n:127.0.0.1:43820_ oasc.Overseer.close Overseer (id=null) closing
[junit4] 2> 1430862 T8006 n:127.0.0.1:43820_ oasc.CoreContainer.initializeAuthenticationPlugin Authentication plugin class obtained from ZK: org.apache.solr.security.KerberosPlugin
[junit4] 2> 1430862 T8006 n:127.0.0.1:43820_ oass.KerberosPlugin.init Params: {token.valid=30, kerberos.principal=HTTP/127.0.0.1, cookie.domain=127.0.0.1, kerberos.keytab=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 9840BAED8B010DDA-001/tempDir-001/minikdc/keytabs, cookie.path=/, type=kerberos, kerberos.name.rules=DEFAULT}
[junit4] 2> 1430864 T8006 n:127.0.0.1:43820_ oasc.CoreContainer.initializeAuthenticationPlugin Reconfiguring the shard handler factory and update shard handler.
[junit4] 2> 1430865 T8006 n:127.0.0.1:43820_ oashc.HttpShardHandlerFactory.reconfigureHttpClient Reconfiguring the default client with: org.apache.solr.client.solrj.impl.Krb5HttpClientConfigurer@370cbbe
[junit4] 2> 1430865 T8006 n:127.0.0.1:43820_ oascsi.Krb5HttpClientConfigurer.configure Setting up SPNego auth with config: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 9840BAED8B010DDA-001/tempDir-001/minikdc/jaas-client.conf
[junit4] 2> 1430865 T8006 n:127.0.0.1:43820_ oasu.UpdateShardHandler.reconfigureHttpClient Reconfiguring the default client with: org.apache.solr.client.solrj.impl.Krb5HttpClientConfigurer@370cbbe
[junit4] 2> 1430866 T8006 n:127.0.0.1:43820_ oascsi.Krb5HttpClientConfigurer.configure Setting up SPNego auth with config: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 9840BAED8B010DDA-001/tempDir-001/minikdc/jaas-client.conf
[junit4] 2> 1430867 T8006 n:127.0.0.1:43820_ oasc.CorePropertiesLocator.discover Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 9840BAED8B010DDA-001/shard-1-001/cores
[junit4] 2> 1430868 T8006 n:127.0.0.1:43820_ oasc.CoreDescriptor.<init> CORE DESCRIPTOR: {name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, loadOnStartup=true, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 9840BAED8B010DDA-001/shard-1-001/cores/collection1, collection=collection1, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 9840BAED8B010DDA-001/shard-1-001/cores/collection1/, coreNodeName=, dataDir=data/, shard=}
[junit4] 2> 1430868 T8006 n:127.0.0.1:43820_ oasc.CorePropertiesLocator.discoverUnder Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 9840BAED8B010DDA-001/shard-1-001/cores/collection1/
[junit4] 2> 1430868 T8006 n:127.0.0.1:43820_ oasc.CorePropertiesLocator.discover Found 1 core definitions
[junit4] 2> 1430871 T8085 n:127.0.0.1:43820_ c:collection1 x:collection1 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
[junit4] 2> 1430871 T8085 n:127.0.0.1:43820_ c:collection1 x:collection1 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 1430872 T8048 n:127.0.0.1:41620_ oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
[junit4] 2> 1430873 T8085 n:127.0.0.1:43820_ oasc.ZkController.preRegister Registering watch for external collection collection1
[junit4] 2> 1430873 T8085 n:127.0.0.1:43820_ oascc.ZkStateReader.addZkWatch addZkWatch collection1
[junit4] 2> 1430873 T8049 n:127.0.0.1:41620_ oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"http://127.0.0.1:43820",
[junit4] 2> "node_name":"127.0.0.1:43820_",
[junit4] 2> "numShards":"2",
[junit4] 2> "state":"down",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "operation":"state"} current state version: 4
[junit4] 2> 1430874 T8085 n:127.0.0.1:43820_ oascc.ZkStateReader.addZkWatch Updating collection state at /collections/collection1/state.json from ZooKeeper...
[junit4] 2> 1430874 T8049 n:127.0.0.1:41620_ oasco.ReplicaMutator.updateState Update state numShards=2 message={
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"http://127.0.0.1:43820",
[junit4] 2> "node_name":"127.0.0.1:43820_",
[junit4] 2> "numShards":"2",
[junit4] 2> "state":"down",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "operation":"state"}
[junit4] 2> 1430874 T8049 n:127.0.0.1:41620_ oasco.ReplicaMutator.updateState Collection already exists with numShards=2
[junit4] 2> 1430875 T8049 n:127.0.0.1:41620_ oasco.ReplicaMutator.updateState Assigning new node to shard shard=shard2
[junit4] 2> 1430875 T8085 n:127.0.0.1:43820_ oascc.ZkStateReader.updateWatchedCollection Updating data for collection1 to ver 0
[junit4] 2> 1430875 T8085 n:127.0.0.1:43820_ oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 1430977 T8049 n:127.0.0.1:41620_ oasco.ZkStateWriter.writePendingUpdates going to update_collection /collections/collection1/state.json version: 0
[junit4] 2> 1430978 T8083 n:127.0.0.1:43820_ oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json for collection collection1 has occurred - updating... (live nodes size: 2)
[junit4] 2> 1430979 T8083 n:127.0.0.1:43820_ oascc.ZkStateReader.updateWatchedCollection Updating data for collection1 to ver 1
[junit4] 2> 1431876 T8085 n:127.0.0.1:43820_ oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 1431876 T8085 n:127.0.0.1:43820_ oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 1431877 T8085 n:127.0.0.1:43820_ oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 1431877 T8085 n:127.0.0.1:43820_ oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 1431878 T8085 n:127.0.0.1:43820_ oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 1431878 T8085 n:127.0.0.1:43820_ oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 9840BAED8B010DDA-001/shard-1-001/cores/collection1/'
[junit4] 2> 1431890 T8085 n:127.0.0.1:43820_ oasc.Config.<init> loaded config solrconfig.xml with version 0
[junit4] 2> 1431895 T8085 n:127.0.0.1:43820_ oasc.SolrConfig.refreshRequestParams current version of requestparams : -1
[junit4] 2> 1431899 T8085 n:127.0.0.1:43820_ oasc.SolrConfig.<init> Using Lucene MatchVersion: 6.0.0
[junit4] 2> 1431920 T8085 n:127.0.0.1:43820_ oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 1431921 T8085 n:127.0.0.1:43820_ oass.IndexSchema.readSchema Reading Solr Schema from /configs/conf1/schema.xml
[junit4] 2> 1431925 T8085 n:127.0.0.1:43820_ oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 1432023 T8085 n:127.0.0.1:43820_ oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 1432029 T8085 n:127.0.0.1:43820_ oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 1432030 T8085 n:127.0.0.1:43820_ oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 1432038 T8085 n:127.0.0.1:43820_ oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 1432041 T8085 n:127.0.0.1:43820_ oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 1432043 T8085 n:127.0.0.1:43820_ oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 1432043 T8085 n:127.0.0.1:43820_ oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 1432044 T8085 n:127.0.0.1:43820_ oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 1432044 T8085 n:127.0.0.1:43820_ oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 1432045 T8085 n:127.0.0.1:43820_ oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 1432045 T8085 n:127.0.0.1:43820_ oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 1432045 T8085 n:127.0.0.1:43820_ oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection collection1
[junit4] 2> 1432045 T8085 n:127.0.0.1:43820_ x:collection1 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 1432045 T8085 n:127.0.0.1:43820_ x:collection1 oasc.SolrCore.<init> [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 9840BAED8B010DDA-001/shard-1-001/cores/collection1/], dataDir=[null]
[junit4] 2> 1432046 T8085 n:127.0.0.1:43820_ x:collection1 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@513effe
[junit4] 2> 1432046 T8085 n:127.0.0.1:43820_ x:collection1 oasc.CachingDirectoryFactory.get return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 9840BAED8B010DDA-001/shard-1-001/cores/collection1/data
[junit4] 2> 1432046 T8085 n:127.0.0.1:43820_ x:collection1 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 9840BAED8B010DDA-001/shard-1-001/cores/collection1/data/index/
[junit4] 2> 1432047 T8085 n:127.0.0.1:43820_ x:collection1 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 9840BAED8B010DDA-001/shard-1-001/cores/collection1/data/index' doesn't exist. Creating new index...
[junit4] 2> 1432047 T8085 n:127.0.0.1:43820_ x:collection1 oasc.CachingDirectoryFactory.get return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 9840BAED8B010DDA-001/shard-1-001/cores/collection1/data/index
[junit4] 2> 1432047 T8085 n:127.0.0.1:43820_ x:collection1 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=10, maxMergeAtOnceExplicit=40, maxMergedSegmentMB=46.45703125, floorSegmentMB=0.359375, forceMergeDeletesPctAllowed=4.1970158864613, segmentsPerTier=34.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
[junit4] 2> 1432048 T8085 n:127.0.0.1:43820_ x:collection1 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@4f84f3ea lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@4fb3db81),segFN=segments_1,generation=1}
[junit4] 2> 1432048 T8085 n:127.0.0.1:43820_ x:collection1 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 1432052 T8085 n:127.0.0.1:43820_ x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 1432052 T8085 n:127.0.0.1:43820_ x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 1432052 T8085 n:127.0.0.1:43820_ x:collection1 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 1432052 T8085 n:127.0.0.1:43820_ x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 1432053 T8085 n:127.0.0.1:43820_ x:collection1 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 1432053 T8085 n:127.0.0.1:43820_ x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 1432053 T8085 n:127.0.0.1:43820_ x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 1432053 T8085 n:127.0.0.1:43820_ x:collection1 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 1432053 T8085 n:127.0.0.1:43820_ x:collection1 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 1432055 T8085 n:127.0.0.1:43820_ x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 1432056 T8085 n:127.0.0.1:43820_ x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 1432057 T8085 n:127.0.0.1:43820_ x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 1432058 T8085 n:127.0.0.1:43820_ x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 1432062 T8085 n:127.0.0.1:43820_ x:collection1 oasc.RequestHandlers.initHandlersFromConfig Registered paths: /admin/mbeans,standard,/update/csv,/update/json/docs,/admin/luke,/admin/segments,/get,/admin/system,/replication,/admin/properties,/config,/schema,/admin/plugins,/admin/logging,/update/json,/admin/threads,/admin/ping,/update,/admin/file
[junit4] 2> 1432062 T8085 n:127.0.0.1:43820_ x:collection1 oasc.SolrCore.initStatsCache Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
[junit4] 2> 1432063 T8085 n:127.0.0.1:43820_ x:collection1 oasu.UpdateHandler.<init> Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 1432063 T8085 n:127.0.0.1:43820_ x:collection1 oasu.UpdateLog.init Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 1432073 T8085 n:127.0.0.1:43820_ x:collection1 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 1432074 T8085 n:127.0.0.1:43820_ x:collection1 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 1432074 T8085 n:127.0.0.1:43820_ x:collection1 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=27, maxMergeAtOnceExplicit=46, maxMergedSegmentMB=61.61328125, floorSegmentMB=1.8525390625, forceMergeDeletesPctAllowed=10.319260276732745, segmentsPerTier=46.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> 1432075 T8085 n:127.0.0.1:43820_ x:collection1 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@4f84f3ea lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@4fb3db81),segFN=segments_1,generation=1}
[junit4] 2> 1432075 T8085 n:127.0.0.1:43820_ x:collection1 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 1432075 T8085 n:127.0.0.1:43820_ x:collection1 oass.SolrIndexSearcher.<init> Opening Searcher@444ca24[collection1] main
[junit4] 2> 1432075 T8085 n:127.0.0.1:43820_ x:collection1 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 1432079 T8085 n:127.0.0.1:43820_ x:collection1 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 1432080 T8085 n:127.0.0.1:43820_ x:collection1 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
[junit4] 2> 1432080 T8085 n:127.0.0.1:43820_ x:collection1 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 1432080 T8085 n:127.0.0.1:43820_ x:collection1 oasr.RestManager.init Initializing RestManager with initArgs: {}
[junit4] 2> 1432080 T8085 n:127.0.0.1:43820_ x:collection1 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1432081 T8085 n:127.0.0.1:43820_ x:collection1 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json
[junit4] 2> 1432081 T8085 n:127.0.0.1:43820_ x:collection1 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1432081 T8085 n:127.0.0.1:43820_ x:collection1 oasr.RestManager.init Initializing 0 registered ManagedResources
[junit4] 2> 1432081 T8085 n:127.0.0.1:43820_ x:collection1 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4] 2> 1432082 T8086 n:127.0.0.1:43820_ x:collection1 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@444ca24[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1432082 T8085 n:127.0.0.1:43820_ x:collection1 oasc.ZkController.getConfDirListeners watch zkdir /configs/conf1
[junit4] 2> 1432083 T8085 n:127.0.0.1:43820_ x:collection1 oasu.UpdateLog.onFirstSearcher On first searcher opened, looking up max value of version field
[junit4] 2> 1432083 T8085 n:127.0.0.1:43820_ x:collection1 oasu.VersionInfo.getMaxVersionFromIndex Refreshing highest value of _version_ for 65536 version buckets from index
[junit4] 2> 1432083 T8085 n:127.0.0.1:43820_ x:collection1 oasu.VersionInfo.getMaxVersionFromIndex No terms found for _version_, cannot seed version bucket highest value from index
[junit4] 2> 1432083 T8085 n:127.0.0.1:43820_ x:collection1 oasu.UpdateLog.seedBucketsWithHighestVersion Could not find max version in index or recent updates, using new clock 1502118470708363264
[junit4] 2> 1432086 T8085 n:127.0.0.1:43820_ x:collection1 oasu.UpdateLog.seedBucketsWithHighestVersion Took 3 ms to seed version buckets with highest version 1502118470708363264
[junit4] 2> 1432086 T8085 n:127.0.0.1:43820_ x:collection1 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 1432087 T8089 n:127.0.0.1:43820_ c:collection1 s:shard2 x:collection1 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:43820 collection:collection1 shard:shard2
[junit4] 2> 1432088 T8006 n:127.0.0.1:43820_ oascsi.Krb5HttpClientConfigurer.configure Setting up SPNego auth with config: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 9840BAED8B010DDA-001/tempDir-001/minikdc/jaas-client.conf
[junit4] 2> 1432088 T8006 n:127.0.0.1:43820_ oass.SolrDispatchFilter.init user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2
[junit4] 2> 1432088 T8006 n:127.0.0.1:43820_ oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 1432089 T8006 oascsi.Krb5HttpClientConfigurer.configure Setting up SPNego auth with config: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 9840BAED8B010DDA-001/tempDir-001/minikdc/jaas-client.conf
[junit4] 2> 1432092 T8089 n:127.0.0.1:43820_ c:collection1 s:shard2 x:collection1 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4] 2> 1432096 T8089 n:127.0.0.1:43820_ c:collection1 s:shard2 x:collection1 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard2
[junit4] 2> 1432097 T8048 n:127.0.0.1:41620_ oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
[junit4] 2> 1432097 T8089 n:127.0.0.1:43820_ c:collection1 s:shard2 x:collection1 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 1432097 T8089 n:127.0.0.1:43820_ c:collection1 s:shard2 x:collection1 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C1287 name=collection1 org.apache.solr.core.SolrCore@aaa5bd9 url=http://127.0.0.1:43820/collection1 node=127.0.0.1:43820_ C1287_STATE=coll:collection1 core:collection1 props:{core=collection1, base_url=http://127.0.0.1:43820, node_name=127.0.0.1:43820_, state=down}
[junit4] 2> 1432098 T8089 n:127.0.0.1:43820_ c:collection1 s:shard2 x:collection1 C1287 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:43820/collection1/
[junit4] 2> 1432098 T8049 n:127.0.0.1:41620_ oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"leader",
[junit4] 2> "shard":"shard2",
[junit4] 2> "collection":"collection1"} current state version: 4
[junit4] 2> 1432098 T8089 n:127.0.0.1:43820_ c:collection1 s:shard2 x:collection1 C1287 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 1432098 T8089 n:127.0.0.1:43820_ c:collection1 s:shard2 x:collection1 C1287 oasc.SyncStrategy.syncToMe http://127.0.0.1:43820/collection1/ has no replicas
[junit4] 2> 1432098 T8089 n:127.0.0.1:43820_ c:collection1 s:shard2 x:collection1 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:43820/collection1/ shard2
[junit4] 2> 1432099 T8089 n:127.0.0.1:43820_ c:collection1 s:shard2 x:collection1 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4] 2> 1432099 T8049 n:127.0.0.1:41620_ oasco.ZkStateWriter.writePendingUpdates going to update_collection /collections/collection1/state.json version: 1
[junit4] 2> 1432099 T8083 n:127.0.0.1:43820_ oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json for collection collection1 has occurred - updating... (live nodes size: 2)
[junit4] 2> 1432101 T8083 n:127.0.0
[...truncated too long message...]
jsh.ContextHandler.doStop Stopped o.e.j.s.ServletContextHandler@6649544c{/,null,UNAVAILABLE}
[junit4] 2> 1448592 T8006 c:control_collection s:shard1 x:collection1 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:57183 57183
[junit4] 2> 1448774 T8014 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:57183 57183
[junit4] 2> 1448775 T8014 oasc.ZkTestServer$ZKServerMain.runFromConfig WARN Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 7 /solr/aliases.json
[junit4] 2> 7 /solr/clusterstate.json
[junit4] 2> 5 /solr/configs/conf1
[junit4] 2> 4 /solr/collections/collection1/state.json
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 7 /solr/live_nodes
[junit4] 2> 5 /solr/overseer/collection-queue-work
[junit4] 2> 4 /solr/overseer/queue
[junit4] 2>
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestSolrCloudWithKerberos -Dtests.method=testKerberizedSolr -Dtests.seed=9840BAED8B010DDA -Dtests.multiplier=2 -Dtests.slow=true -Dtests.locale=ar_TN -Dtests.timezone=Atlantic/Reykjavik -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] ERROR 24.6s J2 | TestSolrCloudWithKerberos.testKerberizedSolr <<<
[junit4] > Throwable #1: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:43820, http://127.0.0.1:45701, http://127.0.0.1:45570, http://127.0.0.1:45919, http://127.0.0.1:41620]
[junit4] > at __randomizedtesting.SeedInfo.seed([9840BAED8B010DDA:33596C2AC160CDED]:0)
[junit4] > at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:355)
[junit4] > at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1086)
[junit4] > at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:856)
[junit4] > at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:799)
[junit4] > at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:135)
[junit4] > at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:152)
[junit4] > at org.apache.solr.cloud.TestSolrCloudWithKerberos.testKerberizedSolr(TestSolrCloudWithKerberos.java:159)
[junit4] > at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:960)
[junit4] > at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:935)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] > Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:41620: Expected mime type application/octet-stream but got text/html. <html>
[junit4] > <head>
[junit4] > <meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/>
[junit4] > <title>Error 500 </title>
[junit4] > </head>
[junit4] > <body>
[junit4] > <h2>HTTP ERROR: 500</h2>
[junit4] > <p>Problem accessing /admin/collections. Reason:
[junit4] > <pre> org.apache.solr.common.SolrException: Error during request authentication, </pre></p>
[junit4] > <hr /><i><small>Powered by Jetty://</small></i>
[junit4] > </body>
[junit4] > </html>
[junit4] > at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:529)
[junit4] > at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:235)
[junit4] > at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:227)
[junit4] > at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:376)
[junit4] > at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:328)
[junit4] > ... 47 more
[junit4] 2> 1448785 T8006 c:control_collection s:shard1 x:collection1 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4] 2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 9840BAED8B010DDA-001
[junit4] 2> 24601 T8005 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene50), sim=DefaultSimilarity, locale=ar_TN, timezone=Atlantic/Reykjavik
[junit4] 2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.8.0_25 (64-bit)/cpus=4,threads=1,free=160314800,total=494927872
[junit4] 2> NOTE: All tests run in this JVM: [TestCloudInspectUtil, TestTrie, OverseerCollectionProcessorTest, CachingDirectoryFactoryTest, TestNoOpRegenerator, DocValuesMissingTest, OpenExchangeRatesOrgProviderTest, SuggesterTest, DirectUpdateHandlerOptimizeTest, TestClusterStateMutator, CursorPagingTest, TestInitParams, HardAutoCommitTest, TestFastOutputStream, TestRawResponseWriter, TestManagedSchemaFieldResource, TestPartialUpdateDeduplication, SolrIndexSplitterTest, TestStressRecovery, AsyncMigrateRouteKeyTest, TestFieldCollectionResource, PrimUtilsTest, ActionThrottleTest, TestExactSharedStatsCache, ResponseLogComponentTest, HdfsRecoverLeaseTest, DeleteShardTest, XmlUpdateRequestHandlerTest, TestSurroundQueryParser, TestSchemaSimilarityResource, TestStandardQParsers, TestQueryUtils, CdcrUpdateLogTest, TestReload, TestMiniSolrCloudClusterSSL, TestUtils, TestReplicationHandlerBackup, MergeStrategyTest, RankQueryTest, SimpleCollectionCreateDeleteTest, TestLuceneMatchVersion, TestQuerySenderNoQuery, TestManagedStopFilterFactory, DistributedQueryElevationComponentTest, BinaryUpdateRequestHandlerTest, OutOfBoxZkACLAndCredentialsProvidersTest, TestReqParamsAPI, TestMacros, PathHierarchyTokenizerFactoryTest, TermVectorComponentDistributedTest, DistributedIntervalFacetingTest, TestRestManager, BasicDistributedZk2Test, DistributedFacetPivotWhiteBoxTest, TestSearcherReuse, FileBasedSpellCheckerTest, DOMUtilTest, OverseerTest, XsltUpdateRequestHandlerTest, TestSolrConfigHandler, TestSolrXml, TestObjectReleaseTracker, TestDefaultSimilarityFactory, TestReRankQParserPlugin, SoftAutoCommitTest, TestMaxScoreQueryParser, RecoveryZkTest, QueryEqualityTest, TestCollationField, FacetPivotSmallTest, CircularListTest, TestNRTOpen, SolrInfoMBeanTest, TestIntervalFaceting, MultiThreadedOCPTest, FullSolrCloudDistribCmdsTest, SimpleFacetsTest, TestCSVResponseWriter, UnloadDistributedZkTest, BasicZkTest, TestRandomFaceting, LeaderElectionTest, TestZkChroot, TestRandomDVFaceting, AutoCommitTest, TestRecovery, TestHashPartitioner, ZkControllerTest, TestStressReorder, TestJoin, TestStressVersions, DistributedTermsComponentTest, TestRangeQuery, SolrCoreTest, StatsComponentTest, SpellCheckComponentTest, TestGroupingSearch, BadIndexSchemaTest, TestFiltering, DirectUpdateHandlerTest, ShowFileRequestHandlerTest, SimplePostToolTest, TestExtendedDismaxParser, SuggesterFSTTest, DocValuesTest, SuggesterWFSTTest, WordBreakSolrSpellCheckerTest, TestPseudoReturnFields, FieldMutatingUpdateProcessorTest, StatelessScriptUpdateProcessorFactoryTest, StandardRequestHandlerTest, PrimitiveFieldTypeTest, TermsComponentTest, TestIndexingPerformance, MoreLikeThisHandlerTest, TestSolrQueryParser, IndexSchemaRuntimeFieldTest, RegexBoostProcessorTest, IndexSchemaTest, JSONWriterTest, TestAnalyzedSuggestions, CSVRequestHandlerTest, TestBinaryResponseWriter, SOLR749Test, CopyFieldTest, TestMergePolicyConfig, TestDocSet, NumericFieldsTest, TestSolrCoreProperties, TestPhraseSuggestions, TestCharFilters, TestCodecSupport, TestXIncludeConfig, TestDFRSimilarityFactory, TestLMDirichletSimilarityFactory, TestFastWriter, ResourceLoaderTest, ChaosMonkeyNothingIsSafeTest, CloudExitableDirectoryReaderTest, ConcurrentDeleteAndCreateCollectionTest, CustomCollectionTest, DistribDocExpirationUpdateProcessorTest, LeaderInitiatedRecoveryOnCommitTest, ReplicationFactorTest, SSLMigrationTest, SaslZkACLProviderTest, TestLeaderElectionZkExpiry, TestMiniSolrCloudCluster, TestReplicaProperties, TestShortCircuitedRequests, TestSolrCloudWithKerberos]
[junit4] Completed [417/502] on J2 in 25.37s, 1 test, 1 error <<< FAILURES!
[...truncated 275 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/build.xml:474: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/build.xml:61: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/extra-targets.xml:39: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build.xml:229: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/common-build.xml:512: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/lucene/common-build.xml:1415: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/lucene/common-build.xml:973: There were test failures: 502 suites, 1984 tests, 1 error, 35 ignored (25 assumptions)
Total time: 49 minutes 36 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
No prior successful build to compare, so performing full copy of artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure