You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Policeman Jenkins Server <je...@thetaphi.de> on 2013/11/12 05:48:06 UTC
[JENKINS] Lucene-Solr-trunk-Linux (32bit/jdk1.7.0_45) - Build #
8278 - Still Failing!
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/8278/
Java: 32bit/jdk1.7.0_45 -client -XX:+UseParallelGC
4 tests failed.
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.DeleteReplicaTest
Error Message:
ERROR: SolrZkClient opens=15 closes=14
Stack Trace:
java.lang.AssertionError: ERROR: SolrZkClient opens=15 closes=14
at __randomizedtesting.SeedInfo.seed([12A5FFB7081C9B8A]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.SolrTestCaseJ4.endTrackingZkClients(SolrTestCaseJ4.java:343)
at org.apache.solr.SolrTestCaseJ4.afterClass(SolrTestCaseJ4.java:135)
at sun.reflect.GeneratedMethodAccessor22.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:700)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:744)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.DeleteReplicaTest
Error Message:
2 threads leaked from SUITE scope at org.apache.solr.cloud.DeleteReplicaTest: 1) Thread[id=3158, name=TEST-DeleteReplicaTest.testDistribSearch-seed#[12A5FFB7081C9B8A]-EventThread, state=WAITING, group=TGRP-DeleteReplicaTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:491) 2) Thread[id=3157, name=TEST-DeleteReplicaTest.testDistribSearch-seed#[12A5FFB7081C9B8A]-SendThread(localhost.localdomain:35791), state=TIMED_WAITING, group=TGRP-DeleteReplicaTest] at java.lang.Thread.sleep(Native Method) at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:86) at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:937) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:993)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 2 threads leaked from SUITE scope at org.apache.solr.cloud.DeleteReplicaTest:
1) Thread[id=3158, name=TEST-DeleteReplicaTest.testDistribSearch-seed#[12A5FFB7081C9B8A]-EventThread, state=WAITING, group=TGRP-DeleteReplicaTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:491)
2) Thread[id=3157, name=TEST-DeleteReplicaTest.testDistribSearch-seed#[12A5FFB7081C9B8A]-SendThread(localhost.localdomain:35791), state=TIMED_WAITING, group=TGRP-DeleteReplicaTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:86)
at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:937)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:993)
at __randomizedtesting.SeedInfo.seed([12A5FFB7081C9B8A]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.DeleteReplicaTest
Error Message:
There are still zombie threads that couldn't be terminated: 1) Thread[id=3157, name=TEST-DeleteReplicaTest.testDistribSearch-seed#[12A5FFB7081C9B8A]-SendThread(localhost.localdomain:35791), state=TIMED_WAITING, group=TGRP-DeleteReplicaTest] at java.lang.Thread.sleep(Native Method) at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:86) at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:937) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:993)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
1) Thread[id=3157, name=TEST-DeleteReplicaTest.testDistribSearch-seed#[12A5FFB7081C9B8A]-SendThread(localhost.localdomain:35791), state=TIMED_WAITING, group=TGRP-DeleteReplicaTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:86)
at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:937)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:993)
at __randomizedtesting.SeedInfo.seed([12A5FFB7081C9B8A]:0)
FAILED: org.apache.solr.cloud.DeleteReplicaTest.testDistribSearch
Error Message:
Invalid replica : core_node3 in shard/collection : shard1/delLiveColl
Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Invalid replica : core_node3 in shard/collection : shard1/delLiveColl
at __randomizedtesting.SeedInfo.seed([12A5FFB7081C9B8A:934371AF7F43FBB6]:0)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:464)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:181)
at org.apache.solr.client.solrj.impl.LBHttpSolrServer.request(LBHttpSolrServer.java:268)
at org.apache.solr.client.solrj.impl.CloudSolrServer.request(CloudSolrServer.java:640)
at org.apache.solr.cloud.DeleteReplicaTest.removeAndWaitForReplicaGone(DeleteReplicaTest.java:149)
at org.apache.solr.cloud.DeleteReplicaTest.deleteLiveReplicaTest(DeleteReplicaTest.java:135)
at org.apache.solr.cloud.DeleteReplicaTest.doTest(DeleteReplicaTest.java:115)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:744)
Build Log:
[...truncated 10411 lines...]
[junit4] Suite: org.apache.solr.cloud.DeleteReplicaTest
[junit4] 2> 688324 T3071 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /euf/
[junit4] 2> 688327 T3071 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-DeleteReplicaTest-1384231497009
[junit4] 2> 688328 T3071 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 2> 688328 T3072 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 688428 T3071 oasc.ZkTestServer.run start zk server on port:35791
[junit4] 2> 688429 T3071 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 688487 T3078 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1c3aacd name:ZooKeeperConnection Watcher:127.0.0.1:35791 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 688487 T3071 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 688488 T3071 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 688492 T3071 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 688493 T3080 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@180e423 name:ZooKeeperConnection Watcher:127.0.0.1:35791/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 688493 T3071 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 688494 T3071 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4] 2> 688497 T3071 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4] 2> 688500 T3071 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4] 2> 688502 T3071 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4] 2> 688504 T3071 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 688505 T3071 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4] 2> 688508 T3071 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4] 2> 688508 T3071 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4] 2> 688511 T3071 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 688511 T3071 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 688513 T3071 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 688513 T3071 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4] 2> 688515 T3071 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 688516 T3071 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4] 2> 688517 T3071 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 688518 T3071 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4] 2> 688520 T3071 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 688520 T3071 oascc.SolrZkClient.makePath makePath: /configs/conf1/enumsConfig.xml
[junit4] 2> 688522 T3071 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 688522 T3071 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4] 2> 688524 T3071 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 688524 T3071 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 688526 T3071 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 688527 T3071 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4] 2> 688528 T3071 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 688529 T3071 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4] 2> 688592 T3071 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 688595 T3071 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:59070
[junit4] 2> 688595 T3071 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 688595 T3071 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 688596 T3071 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.DeleteReplicaTest-controljetty-1384231497212
[junit4] 2> 688596 T3071 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.DeleteReplicaTest-controljetty-1384231497212/'
[junit4] 2> 688608 T3071 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.DeleteReplicaTest-controljetty-1384231497212/solr.xml
[junit4] 2> 688646 T3071 oasc.ConfigSolrXml.<init> Config-defined core root directory:
[junit4] 2> 688647 T3071 oasc.CoreContainer.<init> New CoreContainer 15579368
[junit4] 2> 688647 T3071 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.DeleteReplicaTest-controljetty-1384231497212/]
[junit4] 2> 688648 T3071 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 688649 T3071 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4] 2> 688649 T3071 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 688649 T3071 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 688650 T3071 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 688650 T3071 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 688650 T3071 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 688651 T3071 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 688651 T3071 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 688652 T3071 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 688652 T3071 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 688653 T3071 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:35791/solr
[junit4] 2> 688653 T3071 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 688654 T3071 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 688656 T3091 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1c1c5e4 name:ZooKeeperConnection Watcher:127.0.0.1:35791 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 688656 T3071 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 688658 T3071 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 688659 T3093 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1dc0d07 name:ZooKeeperConnection Watcher:127.0.0.1:35791/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 688660 T3071 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 688661 T3071 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 688664 T3071 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 688666 T3071 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 688667 T3071 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:59070_euf
[junit4] 2> 688668 T3071 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:59070_euf
[junit4] 2> 688670 T3071 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 688674 T3071 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 688675 T3071 oasc.Overseer.start Overseer (id=90716995388112899-127.0.0.1:59070_euf-n_0000000000) starting
[junit4] 2> 688678 T3071 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 688680 T3095 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 688681 T3071 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 688684 T3071 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 688687 T3071 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 688689 T3094 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 688691 T3071 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.DeleteReplicaTest-controljetty-1384231497212
[junit4] 2> 688692 T3071 oasc.CorePropertiesLocator.discoverUnder Found core conf in /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.DeleteReplicaTest-controljetty-1384231497212/conf/
[junit4] 2> 688722 T3071 oasc.CorePropertiesLocator.discover Found 1 core definitions
[junit4] 2> 688723 T3071 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
[junit4] 2> 688723 T3071 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 688725 T3071 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 688727 T3098 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@bbc75b name:ZooKeeperConnection Watcher:127.0.0.1:35791/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 688727 T3071 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 688728 T3071 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 688730 T3071 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 688794 T3071 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 688802 T3071 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:37658
[junit4] 2> 688803 T3071 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 688803 T3071 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 688803 T3071 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.cloud.DeleteReplicaTest-jetty1-1384231497412
[junit4] 2> 688804 T3071 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.DeleteReplicaTest-jetty1-1384231497412/'
[junit4] 2> 688815 T3071 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.DeleteReplicaTest-jetty1-1384231497412/solr.xml
[junit4] 2> 688853 T3071 oasc.ConfigSolrXml.<init> Config-defined core root directory:
[junit4] 2> 688853 T3071 oasc.CoreContainer.<init> New CoreContainer 12241817
[junit4] 2> 688854 T3071 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.cloud.DeleteReplicaTest-jetty1-1384231497412/]
[junit4] 2> 688855 T3071 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 688855 T3071 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4] 2> 688855 T3071 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 688856 T3071 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 688856 T3071 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 688856 T3071 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 688857 T3071 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 688857 T3071 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 688857 T3071 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 688858 T3071 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 688859 T3071 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 688859 T3071 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:35791/solr
[junit4] 2> 688859 T3071 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 688860 T3071 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 688861 T3109 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@19bdef0 name:ZooKeeperConnection Watcher:127.0.0.1:35791 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 688862 T3071 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 688864 T3071 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 688865 T3111 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1f7978c name:ZooKeeperConnection Watcher:127.0.0.1:35791/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 688865 T3071 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 688868 T3071 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 689870 T3071 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:37658_euf
[junit4] 2> 689872 T3071 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:37658_euf
[junit4] 2> 689874 T3093 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 689875 T3111 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 689875 T3098 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 689878 T3071 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.DeleteReplicaTest-jetty1-1384231497412
[junit4] 2> 689879 T3071 oasc.CorePropertiesLocator.discoverUnder Found core conf in /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.DeleteReplicaTest-jetty1-1384231497412/conf/
[junit4] 2> 689902 T3071 oasc.CorePropertiesLocator.discover Found 1 core definitions
[junit4] 2> 689903 T3071 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
[junit4] 2> 689903 T3071 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 689968 T3071 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 689970 T3071 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:37380
[junit4] 2> 689971 T3071 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 689971 T3071 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 689971 T3071 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.DeleteReplicaTest-jetty2-1384231498585
[junit4] 2> 689972 T3071 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.DeleteReplicaTest-jetty2-1384231498585/'
[junit4] 2> 689984 T3071 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.DeleteReplicaTest-jetty2-1384231498585/solr.xml
[junit4] 2> 690021 T3071 oasc.ConfigSolrXml.<init> Config-defined core root directory:
[junit4] 2> 690021 T3071 oasc.CoreContainer.<init> New CoreContainer 2905724
[junit4] 2> 690022 T3071 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.DeleteReplicaTest-jetty2-1384231498585/]
[junit4] 2> 690023 T3071 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 690023 T3071 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4] 2> 690023 T3071 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 690024 T3071 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 690024 T3071 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 690024 T3071 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 690025 T3071 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 690025 T3071 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 690025 T3071 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 690026 T3071 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 690027 T3071 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 690027 T3071 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:35791/solr
[junit4] 2> 690027 T3071 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 690028 T3071 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 690029 T3123 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1d7b36c name:ZooKeeperConnection Watcher:127.0.0.1:35791 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 690030 T3071 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 690032 T3071 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 690033 T3125 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@64d86 name:ZooKeeperConnection Watcher:127.0.0.1:35791/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 690033 T3071 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 690035 T3071 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 691038 T3071 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:37380_euf
[junit4] 2> 691039 T3071 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:37380_euf
[junit4] 2> 691042 T3111 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 691042 T3098 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 691042 T3093 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 691042 T3125 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 691047 T3071 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.DeleteReplicaTest-jetty2-1384231498585
[junit4] 2> 691049 T3071 oasc.CorePropertiesLocator.discoverUnder Found core conf in /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.DeleteReplicaTest-jetty2-1384231498585/conf/
[junit4] 2> 691099 T3071 oasc.CorePropertiesLocator.discover Found 1 core definitions
[junit4] 2> 691100 T3071 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
[junit4] 2> 691100 T3071 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 691165 T3071 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 691167 T3071 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:49044
[junit4] 2> 691167 T3071 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 691168 T3071 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 691168 T3071 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.cloud.DeleteReplicaTest-jetty3-1384231499782
[junit4] 2> 691168 T3071 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.DeleteReplicaTest-jetty3-1384231499782/'
[junit4] 2> 691180 T3071 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.DeleteReplicaTest-jetty3-1384231499782/solr.xml
[junit4] 2> 691218 T3071 oasc.ConfigSolrXml.<init> Config-defined core root directory:
[junit4] 2> 691218 T3071 oasc.CoreContainer.<init> New CoreContainer 11367775
[junit4] 2> 691218 T3071 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.cloud.DeleteReplicaTest-jetty3-1384231499782/]
[junit4] 2> 691220 T3071 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 691220 T3071 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4] 2> 691220 T3071 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 691221 T3071 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 691221 T3071 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 691221 T3071 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 691222 T3071 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 691222 T3071 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 691222 T3071 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 691223 T3071 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 691224 T3071 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 691224 T3071 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:35791/solr
[junit4] 2> 691224 T3071 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 691225 T3071 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 691226 T3137 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1f119b4 name:ZooKeeperConnection Watcher:127.0.0.1:35791 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 691226 T3071 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 691229 T3071 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 691230 T3139 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@417182 name:ZooKeeperConnection Watcher:127.0.0.1:35791/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 691230 T3071 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 691233 T3071 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 692236 T3071 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:49044_euf
[junit4] 2> 692237 T3071 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:49044_euf
[junit4] 2> 692240 T3098 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 692240 T3125 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 692241 T3111 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 692240 T3093 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 692241 T3139 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 692246 T3071 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.DeleteReplicaTest-jetty3-1384231499782
[junit4] 2> 692247 T3071 oasc.CorePropertiesLocator.discoverUnder Found core conf in /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.DeleteReplicaTest-jetty3-1384231499782/conf/
[junit4] 2> 692272 T3071 oasc.CorePropertiesLocator.discover Found 1 core definitions
[junit4] 2> 692273 T3071 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
[junit4] 2> 692273 T3071 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 692339 T3071 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 692341 T3071 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:56359
[junit4] 2> 692341 T3071 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 692342 T3071 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 692342 T3071 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.DeleteReplicaTest-jetty4-1384231500955
[junit4] 2> 692342 T3071 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.DeleteReplicaTest-jetty4-1384231500955/'
[junit4] 2> 692354 T3071 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.DeleteReplicaTest-jetty4-1384231500955/solr.xml
[junit4] 2> 692393 T3071 oasc.ConfigSolrXml.<init> Config-defined core root directory:
[junit4] 2> 692393 T3071 oasc.CoreContainer.<init> New CoreContainer 1348006
[junit4] 2> 692393 T3071 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.DeleteReplicaTest-jetty4-1384231500955/]
[junit4] 2> 692395 T3071 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 692395 T3071 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4] 2> 692395 T3071 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 692396 T3071 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 692396 T3071 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 692396 T3071 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 692397 T3071 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 692397 T3071 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 692397 T3071 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 692398 T3071 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 692399 T3071 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 692399 T3071 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:35791/solr
[junit4] 2> 692399 T3071 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 692400 T3071 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 692401 T3151 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@105adff name:ZooKeeperConnection Watcher:127.0.0.1:35791 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 692401 T3071 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 692403 T3071 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 692404 T3153 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1a5db65 name:ZooKeeperConnection Watcher:127.0.0.1:35791/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 692405 T3071 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 692407 T3071 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 693410 T3071 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:56359_euf
[junit4] 2> 693411 T3071 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:56359_euf
[junit4] 2> 693414 T3093 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4] 2> 693414 T3111 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4] 2> 693414 T3098 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4] 2> 693414 T3125 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4] 2> 693415 T3139 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4] 2> 693415 T3153 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4] 2> 693419 T3071 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.DeleteReplicaTest-jetty4-1384231500955
[junit4] 2> 693420 T3071 oasc.CorePropertiesLocator.discoverUnder Found core conf in /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.DeleteReplicaTest-jetty4-1384231500955/conf/
[junit4] 2> 693458 T3071 oasc.CorePropertiesLocator.discover Found 1 core definitions
[junit4] 2> 693459 T3071 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
[junit4] 2> 693459 T3071 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 693461 T3071 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 693462 T3156 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@86811a name:ZooKeeperConnection Watcher:127.0.0.1:35791/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 693463 T3071 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 693464 T3071 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 693466 T3071 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 693468 T3158 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3bea48 name:ZooKeeperConnection Watcher:127.0.0.1:35791/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 693468 T3071 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 693469 T3071 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 693472 T3086 oasha.CollectionsHandler.handleCreateAction Creating Collection : numShards=2&maxShardsPerNode=2&name=delLiveColl&replicationFactor=2&action=CREATE&wt=javabin&version=2
[junit4] 2> 693473 T3093 oasc.DistributedQueue$LatchChildWatcher.process Watcher fired on path: /overseer/collection-queue-work state: SyncConnected type NodeChildrenChanged
[junit4] 2> 693474 T3095 oasc.OverseerCollectionProcessor.run Overseer Collection Processor: Get the message id:/overseer/collection-queue-work/qn-0000000000 message:{
[junit4] 2> "operation":"createcollection",
[junit4] 2> "fromApi":"true",
[junit4] 2> "name":"delLiveColl",
[junit4] 2> "replicationFactor":"2",
[junit4] 2> "numShards":"2",
[junit4] 2> "maxShardsPerNode":"2"}
[junit4] 2> 693474 T3095 oasc.OverseerCollectionProcessor.processMessage WARN OverseerCollectionProcessor.processMessage : createcollection , {
[junit4] 2> "operation":"createcollection",
[junit4] 2> "fromApi":"true",
[junit4] 2> "name":"delLiveColl",
[junit4] 2> "replicationFactor":"2",
[junit4] 2> "numShards":"2",
[junit4] 2> "maxShardsPerNode":"2"}
[junit4] 2> 694697 T3094 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 694697 T3094 oasc.Overseer$ClusterStateUpdater.buildCollection building a new collection: delLiveColl
[junit4] 2> 694697 T3094 oasc.Overseer$ClusterStateUpdater.createCollection Create collection delLiveColl with shards [shard1, shard2]
[junit4] 2> 694701 T3156 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 694701 T3139 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 694701 T3111 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 694702 T3125 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 694702 T3153 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 694702 T3158 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 694702 T3093 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 694701 T3098 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 694780 T3095 oasc.OverseerCollectionProcessor.createCollection going to create cores replicas shardNames [shard1, shard2] , repFactor : 2
[junit4] 2> 694781 T3095 oasc.OverseerCollectionProcessor.createCollection Creating shard delLiveColl_shard1_replica1 as part of slice shard1 of collection delLiveColl on 127.0.0.1:49044_euf
[junit4] 2> 694781 T3095 oasc.OverseerCollectionProcessor.createCollection Creating shard delLiveColl_shard1_replica2 as part of slice shard1 of collection delLiveColl on 127.0.0.1:56359_euf
[junit4] 2> 694782 T3095 oasc.OverseerCollectionProcessor.createCollection Creating shard delLiveColl_shard2_replica1 as part of slice shard2 of collection delLiveColl on 127.0.0.1:37658_euf
[junit4] 2> 694782 T3095 oasc.OverseerCollectionProcessor.createCollection Creating shard delLiveColl_shard2_replica2 as part of slice shard2 of collection delLiveColl on 127.0.0.1:59070_euf
[junit4] 2> 694783 T3149 oasc.ZkController.publish publishing core=delLiveColl_shard1_replica2 state=down
[junit4] 2> 694784 T3089 oasc.ZkController.publish publishing core=delLiveColl_shard2_replica2 state=down
[junit4] 2> 694784 T3135 oasc.ZkController.publish publishing core=delLiveColl_shard1_replica1 state=down
[junit4] 2> 694783 T3106 oasc.ZkController.publish publishing core=delLiveColl_shard2_replica1 state=down
[junit4] 2> 694785 T3089 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 694786 T3149 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 694786 T3135 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 694786 T3106 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 696206 T3094 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 696207 T3094 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:59070/euf",
[junit4] 2> "core":"delLiveColl_shard2_replica2",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:59070_euf",
[junit4] 2> "shard":"shard2",
[junit4] 2> "shard_range":null,
[junit4] 2> "shard_state":"active",
[junit4] 2> "shard_parent":null,
[junit4] 2> "collection":"delLiveColl",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":null}
[junit4] 2> 696210 T3094 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:49044/euf",
[junit4] 2> "core":"delLiveColl_shard1_replica1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:49044_euf",
[junit4] 2> "shard":"shard1",
[junit4] 2> "shard_range":null,
[junit4] 2> "shard_state":"active",
[junit4] 2> "shard_parent":null,
[junit4] 2> "collection":"delLiveColl",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":null}
[junit4] 2> 696212 T3094 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:56359/euf",
[junit4] 2> "core":"delLiveColl_shard1_replica2",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:56359_euf",
[junit4] 2> "shard":"shard1",
[junit4] 2> "shard_range":null,
[junit4] 2> "shard_state":"active",
[junit4] 2> "shard_parent":null,
[junit4] 2> "collection":"delLiveColl",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":null}
[junit4] 2> 696215 T3094 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:37658/euf",
[junit4] 2> "core":"delLiveColl_shard2_replica1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:37658_euf",
[junit4] 2> "shard":"shard2",
[junit4] 2> "shard_range":null,
[junit4] 2> "shard_state":"active",
[junit4] 2> "shard_parent":null,
[junit4] 2> "collection":"delLiveColl",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":null}
[junit4] 2> 696218 T3156 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 696218 T3111 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 696218 T3158 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 696218 T3153 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 696218 T3139 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 696219 T3093 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 696219 T3098 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 696219 T3125 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 696786 T3089 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for delLiveColl_shard2_replica2
[junit4] 2> 696787 T3135 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for delLiveColl_shard1_replica1
[junit4] 2> 696787 T3106 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for delLiveColl_shard2_replica1
[junit4] 2> 696787 T3089 oasc.CoreContainer.create Creating SolrCore 'delLiveColl_shard2_replica2' using instanceDir: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.DeleteReplicaTest-controljetty-1384231497212/delLiveColl_shard2_replica2
[junit4] 2> 696788 T3135 oasc.CoreContainer.create Creating SolrCore 'delLiveColl_shard1_replica1' using instanceDir: ./org.apache.solr.cloud.DeleteReplicaTest-jetty3-1384231499782/delLiveColl_shard1_replica1
[junit4] 2> 696788 T3135 oasc.ZkController.createCollectionZkNode Check for collection zkNode:delLiveColl
[junit4] 2> 696786 T3149 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for delLiveColl_shard1_replica2
[junit4] 2> 696788 T3106 oasc.CoreContainer.create Creating SolrCore 'delLiveColl_shard2_replica1' using instanceDir: ./org.apache.solr.cloud.DeleteReplicaTest-jetty1-1384231497412/delLiveColl_shard2_replica1
[junit4] 2> 696789 T3106 oasc.ZkController.createCollectionZkNode Check for collection zkNode:delLiveColl
[junit4] 2> 696788 T3089 oasc.ZkController.createCollectionZkNode Check for collection zkNode:delLiveColl
[junit4] 2> 696789 T3149 oasc.CoreContainer.create Creating SolrCore 'delLiveColl_shard1_replica2' using instanceDir: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.DeleteReplicaTest-jetty4-1384231500955/delLiveColl_shard1_replica2
[junit4] 2> 696790 T3089 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:delLiveColl
[junit4] 2> 696789 T3135 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:delLiveColl
[junit4] 2> 696790 T3089 oasc.ZkController.getConfName Looking for collection configName
[junit4] 2> 696790 T3149 oasc.ZkController.createCollectionZkNode Check for collection zkNode:delLiveColl
[junit4] 2> 696790 T3106 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:delLiveColl
[junit4] 2> 696791 T3135 oasc.ZkController.getConfName Looking for collection configName
[junit4] 2> 696792 T3149 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:delLiveColl
[junit4] 2> 696792 T3106 oasc.ZkController.getConfName Looking for collection configName
[junit4] 2> 696792 T3149 oasc.ZkController.getConfName Looking for collection configName
[junit4] 2> 696793 T3135 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
[junit4] 2> 696792 T3089 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
[junit4] 2> 696793 T3106 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
[junit4] 2> 696793 T3135 oascc.SolrZkClient.makePath makePath: /collections/delLiveColl
[junit4] 2> 696794 T3106 oascc.SolrZkClient.makePath makePath: /collections/delLiveColl
[junit4] 2> 696794 T3149 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
[junit4] 2> 696794 T3089 oascc.SolrZkClient.makePath makePath: /collections/delLiveColl
[junit4] 2> 696795 T3149 oascc.SolrZkClient.makePath makePath: /collections/delLiveColl
[junit4] 2> 696796 T3135 oasc.ZkController.readConfigName Load collection config from:/collections/delLiveColl
[junit4] 2> 696796 T3106 oasc.ZkController.readConfigName Load collection config from:/collections/delLiveColl
[junit4] 2> 696797 T3149 oasc.ZkController.readConfigName Load collection config from:/collections/delLiveColl
[junit4] 2> 696797 T3089 oasc.ZkController.readConfigName Load collection config from:/collections/delLiveColl
[junit4] 2> 696798 T3106 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.DeleteReplicaTest-jetty1-1384231497412/delLiveColl_shard2_replica1/'
[junit4] 2> 696798 T3135 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.DeleteReplicaTest-jetty3-1384231499782/delLiveColl_shard1_replica1/'
[junit4] 2> 696798 T3089 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.DeleteReplicaTest-controljetty-1384231497212/delLiveColl_shard2_replica2/'
[junit4] 2> 696799 T3149 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.DeleteReplicaTest-jetty4-1384231500955/delLiveColl_shard1_replica2/'
[junit4] 2> 696853 T3149 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4] 2> 696853 T3135 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4] 2> 696870 T3106 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4] 2> 696886 T3089 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4] 2> 696901 T3149 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 696903 T3149 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 696912 T3149 oass.IndexSchema.readSchema [delLiveColl_shard1_replica2] Schema name=test
[junit4] 2> 696914 T3135 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 696924 T3135 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 696939 T3135 oass.IndexSchema.readSchema [delLiveColl_shard1_replica1] Schema name=test
[junit4] 2> 696956 T3106 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 696957 T3106 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 696966 T3089 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 696968 T3089 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 696971 T3106 oass.IndexSchema.readSchema [delLiveColl_shard2_replica1] Schema name=test
[junit4] 2> 696983 T3089 oass.IndexSchema.readSchema [delLiveColl_shard2_replica2] Schema name=test
[junit4] 2> 697436 T3149 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 697446 T3149 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 697451 T3149 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 697467 T3149 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 697472 T3149 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 697477 T3149 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 697478 T3149 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 697479 T3149 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 697479 T3149 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 697480 T3149 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 697481 T3149 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 697481 T3149 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 697482 T3149 oasc.SolrCore.<init> [delLiveColl_shard1_replica2] Opening new SolrCore at ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.DeleteReplicaTest-jetty4-1384231500955/delLiveColl_shard1_replica2/, dataDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.DeleteReplicaTest-jetty4-1384231500955/delLiveColl_shard1_replica2/data/
[junit4] 2> 697482 T3149 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@13cbac2
[junit4] 2> 697483 T3149 oasc.CachingDirectoryFactory.get return new directory for ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.DeleteReplicaTest-jetty4-1384231500955/delLiveColl_shard1_replica2/data
[junit4] 2> 697484 T3149 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.DeleteReplicaTest-jetty4-1384231500955/delLiveColl_shard1_replica2/data/index/
[junit4] 2> 697484 T3149 oasc.SolrCore.initIndex WARN [delLiveColl_shard1_replica2] Solr index directory '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.DeleteReplicaTest-jetty4-1384231500955/delLiveColl_shard1_replica2/data/index' doesn't exist. Creating new index...
[junit4] 2> 697485 T3149 oasc.CachingDirectoryFactory.get return new directory for ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.DeleteReplicaTest-jetty4-1384231500955/delLiveColl_shard1_replica2/data/index
[junit4] 2> 697485 T3149 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=863926986, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
[junit4] 2> 697486 T3149 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@18f4b71 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1fa4977),segFN=segments_1,generation=1}
[junit4] 2> 697487 T3149 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 697491 T3149 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 697491 T3149 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 697492 T3149 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 697492 T3149 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 697493 T3149 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 697493 T3149 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 697494 T3149 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 697494 T3149 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 697495 T3149 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 697496 T3149 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 697496 T3149 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 697497 T3149 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 697497 T3149 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 697498 T3149 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 697499 T3149 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 697502 T3149 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 697507 T3149 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 697507 T3149 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 697508 T3149 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=11, maxMergeAtOnceExplicit=14, maxMergedSegmentMB=12.4794921875, floorSegmentMB=1.3798828125, forceMergeDeletesPctAllowed=23.41329550237682, segmentsPerTier=14.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> 697509 T3149 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@18f4b71 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1fa4977),segFN=segments_1,generation=1}
[junit4] 2> 697510 T3149 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 697510 T3149 oass.SolrIndexSearcher.<init> Opening Searcher@181b2be main
[junit4] 2> 697515 T3163 oasc.SolrCore.registerSearcher [delLiveColl_shard1_replica2] Registered new searcher Searcher@181b2be main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4] 2> 697516 T3149 oasc.CoreContainer.registerCore registering core: delLiveColl_shard1_replica2
[junit4] 2> 697517 T3149 oasc.ZkController.register Register replica - core:delLiveColl_shard1_replica2 address:http://127.0.0.1:56359/euf collection:delLiveColl shard:shard1
[junit4] 2> 697518 T3149 oascc.SolrZkClient.makePath makePath: /collections/delLiveColl/leader_elect/shard1/election
[junit4] 2> 697527 T3149 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 697530 T3149 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Waiting until we see more replicas up for shard shard1: total=2 found=1 timeoutin=180000
[junit4] 2> 697565 T3089 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 697575 T3089 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 697580 T3089 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 697592 T3089 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 697597 T3089 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 697601 T3089 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 697603 T3089 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 697603 T3089 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 697604 T3089 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 697605 T3089 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 697605 T3089 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 697606 T3089 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 697606 T3089 oasc.SolrCore.<init> [delLiveColl_shard2_replica2] Opening new SolrCore at ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.DeleteReplicaTest-controljetty-1384231497212/delLiveColl_shard2_replica2/, dataDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.DeleteReplicaTest-controljetty-1384231497212/delLiveColl_shard2_replica2/data/
[junit4] 2> 697607 T3089 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@13cbac2
[junit4] 2> 697608 T3089 oasc.CachingDirectoryFactory.get return new directory for ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.DeleteReplicaTest-controljetty-1384231497212/delLiveColl_shard2_replica2/data
[junit4] 2> 697608 T3089 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.DeleteReplicaTest-controljetty-1384231497212/delLiveColl_shard2_replica2/data/index/
[junit4] 2> 697609 T3089 oasc.SolrCore.initIndex WARN [delLiveColl_shard2_replica2] Solr index directory '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.DeleteReplicaTest-controljetty-1384231497212/delLiveColl_shard2_replica2/data/index' doesn't exist. Creating new index...
[junit4] 2> 697609 T3089 oasc.CachingDirectoryFactory.get return new directory for ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.DeleteReplicaTest-controljetty-1384231497212/delLiveColl_shard2_replica2/data/index
[junit4] 2> 697610 T3089 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=863926986, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
[junit4] 2> 697611 T3089 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@2ce9ea lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1543926),segFN=segments_1,generation=1}
[junit4] 2> 697611 T3089 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 697615 T3089 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 697616 T3089 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 697617 T3089 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 697617 T3089 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 697617 T3089 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 697618 T3089 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 697619 T3089 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 697619 T3089 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 697619 T3089 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 697620 T3089 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 697621 T3089 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 697621 T3089 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 697622 T3089 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 697623 T3089 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 697623 T3089 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 697625 T3089 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 697630 T3089 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 697630 T3089 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 697631 T3089 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=11, maxMergeAtOnceExplicit=14, maxMergedSegmentMB=12.4794921875, floorSegmentMB=1.3798828125, forceMergeDeletesPctAllowed=23.41329550237682, segmentsPerTier=14.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> 697632 T3089 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@2ce9ea lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1543926),segFN=segments_1,generation=1}
[junit4] 2> 697633 T3089 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 697633 T3089 oass.SolrIndexSearcher.<init> Opening Searcher@eb8a72 main
[junit4] 2> 697638 T3164 oasc.SolrCore.registerSearcher [delLiveColl_shard2_replica2] Registered new searcher Searcher@eb8a72 main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4] 2> 697640 T3089 oasc.CoreContainer.registerCore registering core: delLiveColl_shard2_replica2
[junit4] 2> 697641 T3089 oasc.ZkController.register Register replica - core:delLiveColl_shard2_replica2 address:http://127.0.0.1:59070/euf collection:delLiveColl shard:shard2
[junit4] 2> 697642 T3089 oascc.SolrZkClient.makePath makePath: /collections/delLiveColl/leader_elect/shard2/election
[junit4] 2> 697652 T3089 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard2
[junit4] 2> 697653 T3135 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 697655 T3089 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Waiting until we see more replicas up for shard shard2: total=2 found=1 timeoutin=179999
[junit4] 2> 697659 T3135 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 697662 T3106 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 697662 T3135 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 697671 T3106 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 697672 T3135 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 697675 T3135 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 697676 T3106 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 697679 T3135 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 697681 T3135 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 697681 T3135 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 697682 T3135 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 697683 T3135 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 697684 T3135 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 697684 T3135 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 697685 T3135 oasc.SolrCore.<init> [delLiveColl_shard1_replica1] Opening new SolrCore at ./org.apache.solr.cloud.DeleteReplicaTest-jetty3-1384231499782/delLiveColl_shard1_replica1/, dataDir=./org.apache.solr.cloud.DeleteReplicaTest-jetty3-1384231499782/delLiveColl_shard1_replica1/data/
[junit4] 2> 697685 T3135 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@13cbac2
[junit4] 2> 697686 T3135 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.DeleteReplicaTest-jetty3-1384231499782/delLiveColl_shard1_replica1/data
[junit4] 2> 697687 T3135 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.DeleteReplicaTest-jetty3-1384231499782/delLiveColl_shard1_replica1/data/index/
[junit4] 2> 697687 T3135 oasc.SolrCore.initIndex WARN [delLiveColl_shard1_replica1] Solr index directory './org.apache.solr.cloud.DeleteReplicaTest-jetty3-1384231499782/delLiveColl_shard1_replica1/data/index' doesn't exist. Creating new index...
[junit4] 2> 697688 T3135 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.DeleteReplicaTest-jetty3-1384231499782/delLiveColl_shard1_replica1/data/index
[junit4] 2> 697688 T3135 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=863926986, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
[junit4] 2> 697689 T3135 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@5f9b24 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1958d48),segFN=segments_1,generation=1}
[junit4] 2> 697690 T3135 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 697691 T3106 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 697695 T3135 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 697695 T3135 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 697696 T3106 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 697696 T3135 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 697697 T3135 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 697697 T3135 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 697698 T3135 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 697698 T3135 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 697699 T3135 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 697699 T3135 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 697700 T3135 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 697701 T3135 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 697701 T3135 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 697701 T3106 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 697702 T3135 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 697703 T3106 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 697703 T3106 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 697703 T3135 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 697704 T3106 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 697705 T3135 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 697705 T3106 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 697706 T3106 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 697706 T3106 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 697707 T3106 oasc.SolrCore.<init> [delLiveColl_shard2_replica1] Opening new SolrCore at ./org.apache.solr.cloud.DeleteReplicaTest-jetty1-1384231497412/delLiveColl_shard2_replica1/, dataDir=./org.apache.solr.cloud.DeleteReplicaTest-jetty1-1384231497412/delLiveColl_shard2_replica1/data/
[junit4] 2> 697707 T3135 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 697707 T3106 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@13cbac2
[junit4] 2> 697709 T3106 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.DeleteReplicaTest-jetty1-1384231497412/delLiveColl_shard2_replica1/data
[junit4] 2> 697709 T3106 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.DeleteReplicaTest-jetty1-1384231497412/delLiveColl_shard2_replica1/data/index/
[junit4] 2> 697709 T3106 oasc.SolrCore.initIndex WARN [delLiveColl_shard2_replica1] Solr index directory './org.apache.solr.cloud.DeleteReplicaTest-jetty1-1384231497412/delLiveColl_shard2_replica1/data/index' doesn't exist. Creating new index...
[junit4] 2> 697710 T3106 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.DeleteReplicaTest-jetty1-1384231497412/delLiveColl_shard2_replica1/data/index
[junit4] 2> 697710 T3135 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 697711 T3106 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=863926986, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
[junit4] 2> 697711 T3135 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 697712 T3106 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@17a219f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@f0abef),segFN=segments_1,generation=1}
[junit4] 2> 697712 T3135 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=11, maxMergeAtOnceExplicit=14, maxMergedSegmentMB=12.4794921875, floorSegmentMB=1.3798828125, forceMergeDeletesPctAllowed=23.41329550237682, segmentsPerTier=14.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> 697712 T3106 oasc.SolrDeletionPolicy.updateC
[...truncated too long message...]
ative Method)
[junit4] 2> at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:735)
[junit4] 2> at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
[junit4] 2>
[junit4] 2> 813374 T3157 oazc.StaticHostProvider.next WARN Unexpected exception java.lang.InterruptedException: sleep interrupted
[junit4] 2> at java.lang.Thread.sleep(Native Method)
[junit4] 2> at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:86)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:937)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:993)
[junit4] 2>
[junit4] 2> 813375 T3157 oaz.ClientCnxn$SendThread.run WARN Session 0x1424aa06d33000e for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused
[junit4] 2> at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
[junit4] 2> at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:735)
[junit4] 2> at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
[junit4] 2>
[junit4] 2> 813874 T3157 oazc.StaticHostProvider.next WARN Unexpected exception java.lang.InterruptedException: sleep interrupted
[junit4] 2> at java.lang.Thread.sleep(Native Method)
[junit4] 2> at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:86)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:937)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:993)
[junit4] 2>
[junit4] 2> 813875 T3157 oaz.ClientCnxn$SendThread.run WARN Session 0x1424aa06d33000e for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused
[junit4] 2> at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
[junit4] 2> at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:735)
[junit4] 2> at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
[junit4] 2>
[junit4] 2> 814374 T3157 oaz.ClientCnxn$SendThread.run WARN Unexpected exception java.lang.InterruptedException: sleep interrupted
[junit4] 2> at java.lang.Thread.sleep(Native Method)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:984)
[junit4] 2>
[junit4] 2> 126551 T3070 ccr.ThreadLeakControl.tryToInterruptAll SEVERE There are still zombie threads that couldn't be terminated:
[junit4] 2> 1) Thread[id=3157, name=TEST-DeleteReplicaTest.testDistribSearch-seed#[12A5FFB7081C9B8A]-SendThread(localhost.localdomain:35791), state=TIMED_WAITING, group=TGRP-DeleteReplicaTest]
[junit4] 2> at java.lang.Thread.sleep(Native Method)
[junit4] 2> at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:86)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:937)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:993)
[junit4] 2> NOTE: test params are: codec=Asserting, sim=RandomSimilarityProvider(queryNorm=false,coord=yes): {}, locale=ar_DZ, timezone=GMT0
[junit4] 2> NOTE: Linux 3.8.0-32-generic i386/Oracle Corporation 1.7.0_45 (32-bit)/cpus=8,threads=3,free=168394784,total=309592064
[junit4] 2> NOTE: All tests run in this JVM: [TestJoin, TestPostingsSolrHighlighter, TestQueryUtils, MBeansHandlerTest, QueryEqualityTest, TestSolrDeletionPolicy1, TestImplicitCoreProperties, DistributedQueryElevationComponentTest, TestPseudoReturnFields, TestReplicationHandler, ClusterStateTest, TestFieldTypeCollectionResource, URLClassifyProcessorTest, RecoveryZkTest, PreAnalyzedUpdateProcessorTest, ZkCLITest, TestPerFieldSimilarity, BadComponentTest, AnalysisAfterCoreReloadTest, ShardRoutingTest, JsonLoaderTest, SliceStateUpdateTest, AddBlockUpdateTest, TestCopyFieldCollectionResource, TestNRTOpen, TestSolrQueryParser, TestValueSourceCache, UpdateRequestProcessorFactoryTest, TestSolrXmlPersistor, CircularListTest, TestWriterPerf, StatelessScriptUpdateProcessorFactoryTest, TestTrie, TestRandomDVFaceting, BadCopyFieldTest, TestFieldCollectionResource, TestReloadAndDeleteDocs, TestFieldTypeResource, CoreAdminHandlerTest, TestLMDirichletSimilarityFactory, TestFunctionQuery, TermsComponentTest, TestMergePolicyConfig, QueryParsingTest, TestPHPSerializedResponseWriter, ZkControllerTest, SolrIndexSplitterTest, MoreLikeThisHandlerTest, DistributedTermsComponentTest, TestRealTimeGet, TestLuceneMatchVersion, TestElisionMultitermQuery, TestDocSet, SolrXmlInZkTest, UpdateParamsTest, CopyFieldTest, ScriptEngineTest, IndexSchemaTest, SyncSliceTest, ExternalFileFieldSortTest, TestExtendedDismaxParser, EchoParamsTest, TestCloudManagedSchema, CustomCollectionTest, HdfsRecoveryZkTest, TestClassNameShortening, TestSchemaNameResource, DistributedSpellCheckComponentTest, TestCharFilters, TestSolrXmlPersistence, FieldMutatingUpdateProcessorTest, SimpleFacetsTest, LeaderElectionTest, TestManagedSchema, LoggingHandlerTest, AutoCommitTest, QueryElevationComponentTest, FastVectorHighlighterTest, CacheHeaderTest, TestSolrIndexConfig, CSVRequestHandlerTest, DeleteReplicaTest]
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=DeleteReplicaTest -Dtests.seed=12A5FFB7081C9B8A -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=ar_DZ -Dtests.timezone=GMT0 -Dtests.file.encoding=US-ASCII
[junit4] ERROR 0.00s J0 | DeleteReplicaTest (suite) <<<
[junit4] > Throwable #1: java.lang.AssertionError: ERROR: SolrZkClient opens=15 closes=14
[junit4] > at __randomizedtesting.SeedInfo.seed([12A5FFB7081C9B8A]:0)
[junit4] > at org.apache.solr.SolrTestCaseJ4.endTrackingZkClients(SolrTestCaseJ4.java:343)
[junit4] > at org.apache.solr.SolrTestCaseJ4.afterClass(SolrTestCaseJ4.java:135)
[junit4] > at java.lang.Thread.run(Thread.java:744)Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: 2 threads leaked from SUITE scope at org.apache.solr.cloud.DeleteReplicaTest:
[junit4] > 1) Thread[id=3158, name=TEST-DeleteReplicaTest.testDistribSearch-seed#[12A5FFB7081C9B8A]-EventThread, state=WAITING, group=TGRP-DeleteReplicaTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
[junit4] > at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4] > at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:491)
[junit4] > 2) Thread[id=3157, name=TEST-DeleteReplicaTest.testDistribSearch-seed#[12A5FFB7081C9B8A]-SendThread(localhost.localdomain:35791), state=TIMED_WAITING, group=TGRP-DeleteReplicaTest]
[junit4] > at java.lang.Thread.sleep(Native Method)
[junit4] > at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:86)
[junit4] > at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:937)
[junit4] > at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:993)
[junit4] > at __randomizedtesting.SeedInfo.seed([12A5FFB7081C9B8A]:0)Throwable #3: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
[junit4] > 1) Thread[id=3157, name=TEST-DeleteReplicaTest.testDistribSearch-seed#[12A5FFB7081C9B8A]-SendThread(localhost.localdomain:35791), state=TIMED_WAITING, group=TGRP-DeleteReplicaTest]
[junit4] > at java.lang.Thread.sleep(Native Method)
[junit4] > at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:86)
[junit4] > at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:937)
[junit4] > at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:993)
[junit4] > at __randomizedtesting.SeedInfo.seed([12A5FFB7081C9B8A]:0)
[junit4] Completed on J0 in 126.59s, 1 test, 1 failure, 3 errors <<< FAILURES!
[...truncated 504 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:423: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:403: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:39: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build.xml:189: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/common-build.xml:489: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:1271: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:907: There were test failures: 334 suites, 1470 tests, 3 suite-level errors, 1 error, 624 ignored (2 assumptions)
Total time: 35 minutes 59 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 32bit/jdk1.7.0_45 -client -XX:+UseParallelGC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure