You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Policeman Jenkins Server <je...@thetaphi.de> on 2013/11/07 22:54:01 UTC
[JENKINS] Lucene-Solr-trunk-MacOSX (64bit/jdk1.7.0) - Build # 995 -
Still Failing!
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-MacOSX/995/
Java: 64bit/jdk1.7.0 -XX:+UseCompressedOops -XX:+UseG1GC
4 tests failed.
FAILED: org.apache.solr.cloud.DeleteReplicaTest.testDistribSearch
Error Message:
Invalid replica : core_node1 in shard/collection : shard1/delLiveColl
Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Invalid replica : core_node1 in shard/collection : shard1/delLiveColl
at __randomizedtesting.SeedInfo.seed([70519D0640F75D32:F1B7131E37A83D0E]:0)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:464)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:181)
at org.apache.solr.client.solrj.impl.LBHttpSolrServer.request(LBHttpSolrServer.java:268)
at org.apache.solr.client.solrj.impl.CloudSolrServer.request(CloudSolrServer.java:640)
at org.apache.solr.cloud.DeleteReplicaTest.removeAndWaitForReplicaGone(DeleteReplicaTest.java:149)
at org.apache.solr.cloud.DeleteReplicaTest.deleteLiveReplicaTest(DeleteReplicaTest.java:135)
at org.apache.solr.cloud.DeleteReplicaTest.doTest(DeleteReplicaTest.java:115)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
at sun.reflect.GeneratedMethodAccessor53.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:744)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.DeleteReplicaTest
Error Message:
ERROR: SolrZkClient opens=15 closes=14
Stack Trace:
java.lang.AssertionError: ERROR: SolrZkClient opens=15 closes=14
at __randomizedtesting.SeedInfo.seed([70519D0640F75D32]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.SolrTestCaseJ4.endTrackingZkClients(SolrTestCaseJ4.java:343)
at org.apache.solr.SolrTestCaseJ4.afterClass(SolrTestCaseJ4.java:135)
at sun.reflect.GeneratedMethodAccessor23.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:700)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:744)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.DeleteReplicaTest
Error Message:
2 threads leaked from SUITE scope at org.apache.solr.cloud.DeleteReplicaTest: 1) Thread[id=8176, name=TEST-DeleteReplicaTest.testDistribSearch-seed#[70519D0640F75D32]-EventThread, state=WAITING, group=TGRP-DeleteReplicaTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:491) 2) Thread[id=8175, name=TEST-DeleteReplicaTest.testDistribSearch-seed#[70519D0640F75D32]-SendThread(localhost:51527), state=TIMED_WAITING, group=TGRP-DeleteReplicaTest] at java.lang.Thread.sleep(Native Method) at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:86) at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:937) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:993)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 2 threads leaked from SUITE scope at org.apache.solr.cloud.DeleteReplicaTest:
1) Thread[id=8176, name=TEST-DeleteReplicaTest.testDistribSearch-seed#[70519D0640F75D32]-EventThread, state=WAITING, group=TGRP-DeleteReplicaTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:491)
2) Thread[id=8175, name=TEST-DeleteReplicaTest.testDistribSearch-seed#[70519D0640F75D32]-SendThread(localhost:51527), state=TIMED_WAITING, group=TGRP-DeleteReplicaTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:86)
at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:937)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:993)
at __randomizedtesting.SeedInfo.seed([70519D0640F75D32]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.DeleteReplicaTest
Error Message:
There are still zombie threads that couldn't be terminated: 1) Thread[id=8175, name=TEST-DeleteReplicaTest.testDistribSearch-seed#[70519D0640F75D32]-SendThread(localhost:51527), state=TIMED_WAITING, group=TGRP-DeleteReplicaTest] at java.lang.Thread.sleep(Native Method) at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:86) at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:937) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:993)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
1) Thread[id=8175, name=TEST-DeleteReplicaTest.testDistribSearch-seed#[70519D0640F75D32]-SendThread(localhost:51527), state=TIMED_WAITING, group=TGRP-DeleteReplicaTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:86)
at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:937)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:993)
at __randomizedtesting.SeedInfo.seed([70519D0640F75D32]:0)
Build Log:
[...truncated 10648 lines...]
[junit4] Suite: org.apache.solr.cloud.DeleteReplicaTest
[junit4] 2> 2838183 T8089 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /n_ok/
[junit4] 2> 2838189 T8089 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4] 2> Creating dataDir: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./solrtest-DeleteReplicaTest-1383861068962
[junit4] 2> 2838191 T8089 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 2> 2838192 T8090 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 2838293 T8089 oasc.ZkTestServer.run start zk server on port:51527
[junit4] 2> 2838295 T8089 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 2838304 T8096 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@23136f07 name:ZooKeeperConnection Watcher:127.0.0.1:51527 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 2838305 T8089 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 2838305 T8089 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 2838321 T8089 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 2838327 T8098 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@9497c07 name:ZooKeeperConnection Watcher:127.0.0.1:51527/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 2838327 T8089 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 2838328 T8089 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4] 2> 2838338 T8089 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4] 2> 2838349 T8089 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4] 2> 2838356 T8089 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4] 2> 2838364 T8089 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 2838365 T8089 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4] 2> 2838378 T8089 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4] 2> 2838378 T8089 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4] 2> 2838387 T8089 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 2838388 T8089 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 2838396 T8089 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 2838397 T8089 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4] 2> 2838407 T8089 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 2838408 T8089 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4] 2> 2838416 T8089 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 2838417 T8089 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4] 2> 2838426 T8089 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 2838427 T8089 oascc.SolrZkClient.makePath makePath: /configs/conf1/enumsConfig.xml
[junit4] 2> 2838436 T8089 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 2838437 T8089 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4] 2> 2838445 T8089 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 2838446 T8089 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 2838455 T8089 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 2838456 T8089 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4] 2> 2838464 T8089 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 2838465 T8089 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4] 2> 2838803 T8089 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 2838812 T8089 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:51530
[junit4] 2> 2838813 T8089 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 2838813 T8089 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 2838814 T8089 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.DeleteReplicaTest-controljetty-1383861069247
[junit4] 2> 2838814 T8089 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.DeleteReplicaTest-controljetty-1383861069247/'
[junit4] 2> 2838869 T8089 oasc.ConfigSolr.fromFile Loading container configuration from /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.DeleteReplicaTest-controljetty-1383861069247/solr.xml
[junit4] 2> 2838983 T8089 oasc.ConfigSolrXml.<init> Config-defined core root directory:
[junit4] 2> 2838983 T8089 oasc.CoreContainer.<init> New CoreContainer 1418835704
[junit4] 2> 2838984 T8089 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.DeleteReplicaTest-controljetty-1383861069247/]
[junit4] 2> 2838987 T8089 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 2838988 T8089 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4] 2> 2838988 T8089 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 2838989 T8089 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 2838990 T8089 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 2838990 T8089 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 2838990 T8089 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 2838991 T8089 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 2838992 T8089 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 2838996 T8089 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 2838997 T8089 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 2838997 T8089 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:51527/solr
[junit4] 2> 2838997 T8089 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 2839000 T8089 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 2839005 T8109 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@21a15bf9 name:ZooKeeperConnection Watcher:127.0.0.1:51527 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 2839006 T8089 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 2839012 T8089 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 2839017 T8111 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6ceba945 name:ZooKeeperConnection Watcher:127.0.0.1:51527/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 2839018 T8089 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 2839023 T8089 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 2839034 T8089 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 2839043 T8089 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 2839048 T8089 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:51530_n_ok
[junit4] 2> 2839051 T8089 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:51530_n_ok
[junit4] 2> 2839060 T8089 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 2839076 T8089 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 2839082 T8089 oasc.Overseer.start Overseer (id=90692719016083459-127.0.0.1:51530_n_ok-n_0000000000) starting
[junit4] 2> 2839092 T8089 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 2839103 T8113 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 2839104 T8089 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 2839110 T8089 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 2839116 T8089 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 2839250 T8112 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 2839592 T8089 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.DeleteReplicaTest-controljetty-1383861069247
[junit4] 2> 2839737 T8089 oasc.CorePropertiesLocator.discoverUnder Found core conf in /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.DeleteReplicaTest-controljetty-1383861069247/conf/
[junit4] 2> 2839748 T8089 oasc.CorePropertiesLocator.discover Found 1 core definitions
[junit4] 2> 2839750 T8089 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0
[junit4] 2> 2839751 T8089 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 2839757 T8089 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 2839763 T8116 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1cbe1830 name:ZooKeeperConnection Watcher:127.0.0.1:51527/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 2839764 T8089 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 2839767 T8089 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 2839776 T8089 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 2840143 T8089 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 2840148 T8089 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:51534
[junit4] 2> 2840149 T8089 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 2840149 T8089 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 2840150 T8089 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.cloud.DeleteReplicaTest-jetty1-1383861070549
[junit4] 2> 2840150 T8089 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.DeleteReplicaTest-jetty1-1383861070549/'
[junit4] 2> 2840194 T8089 oasc.ConfigSolr.fromFile Loading container configuration from /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.DeleteReplicaTest-jetty1-1383861070549/solr.xml
[junit4] 2> 2840271 T8089 oasc.ConfigSolrXml.<init> Config-defined core root directory:
[junit4] 2> 2840272 T8089 oasc.CoreContainer.<init> New CoreContainer 513557467
[junit4] 2> 2840273 T8089 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.cloud.DeleteReplicaTest-jetty1-1383861070549/]
[junit4] 2> 2840276 T8089 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 2840276 T8089 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4] 2> 2840276 T8089 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 2840277 T8089 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 2840277 T8089 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 2840278 T8089 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 2840278 T8089 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 2840278 T8089 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 2840278 T8089 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 2840283 T8089 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 2840283 T8089 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 2840283 T8089 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:51527/solr
[junit4] 2> 2840284 T8089 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 2840286 T8089 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 2840291 T8127 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@28bac846 name:ZooKeeperConnection Watcher:127.0.0.1:51527 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 2840292 T8089 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 2840298 T8089 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 2840303 T8129 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@28392bee name:ZooKeeperConnection Watcher:127.0.0.1:51527/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 2840303 T8089 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 2840314 T8089 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 2841323 T8089 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:51534_n_ok
[junit4] 2> 2841332 T8089 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:51534_n_ok
[junit4] 2> 2841340 T8129 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 2841340 T8111 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 2841340 T8116 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 2841354 T8089 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.DeleteReplicaTest-jetty1-1383861070549
[junit4] 2> 2841483 T8089 oasc.CorePropertiesLocator.discoverUnder Found core conf in /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.DeleteReplicaTest-jetty1-1383861070549/conf/
[junit4] 2> 2841491 T8089 oasc.CorePropertiesLocator.discover Found 1 core definitions
[junit4] 2> 2841492 T8089 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0
[junit4] 2> 2841493 T8089 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 2841832 T8089 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 2841836 T8089 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:51537
[junit4] 2> 2841836 T8089 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 2841838 T8089 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 2841838 T8089 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.cloud.DeleteReplicaTest-jetty2-1383861072270
[junit4] 2> 2841838 T8089 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.DeleteReplicaTest-jetty2-1383861072270/'
[junit4] 2> 2841877 T8089 oasc.ConfigSolr.fromFile Loading container configuration from /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.DeleteReplicaTest-jetty2-1383861072270/solr.xml
[junit4] 2> 2841961 T8089 oasc.ConfigSolrXml.<init> Config-defined core root directory:
[junit4] 2> 2841962 T8089 oasc.CoreContainer.<init> New CoreContainer 1864604169
[junit4] 2> 2841962 T8089 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.cloud.DeleteReplicaTest-jetty2-1383861072270/]
[junit4] 2> 2841964 T8089 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 2841964 T8089 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4] 2> 2841964 T8089 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 2841965 T8089 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 2841965 T8089 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 2841965 T8089 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 2841966 T8089 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 2841967 T8089 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 2841967 T8089 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 2841971 T8089 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 2841971 T8089 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 2841972 T8089 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:51527/solr
[junit4] 2> 2841972 T8089 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 2841973 T8089 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 2841978 T8141 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7114cc22 name:ZooKeeperConnection Watcher:127.0.0.1:51527 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 2841978 T8089 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 2841984 T8089 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 2841988 T8143 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@53ec2a74 name:ZooKeeperConnection Watcher:127.0.0.1:51527/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 2841989 T8089 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 2842000 T8089 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 2843011 T8089 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:51537_n_ok
[junit4] 2> 2843014 T8089 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:51537_n_ok
[junit4] 2> 2843021 T8129 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 2843022 T8111 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 2843023 T8116 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 2843023 T8143 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 2843034 T8089 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.DeleteReplicaTest-jetty2-1383861072270
[junit4] 2> 2843128 T8089 oasc.CorePropertiesLocator.discoverUnder Found core conf in /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.DeleteReplicaTest-jetty2-1383861072270/conf/
[junit4] 2> 2843134 T8089 oasc.CorePropertiesLocator.discover Found 1 core definitions
[junit4] 2> 2843135 T8089 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0
[junit4] 2> 2843135 T8089 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 2843454 T8089 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 2843458 T8089 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:51540
[junit4] 2> 2843458 T8089 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 2843459 T8089 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 2843459 T8089 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.DeleteReplicaTest-jetty3-1383861073911
[junit4] 2> 2843459 T8089 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.DeleteReplicaTest-jetty3-1383861073911/'
[junit4] 2> 2843500 T8089 oasc.ConfigSolr.fromFile Loading container configuration from /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.DeleteReplicaTest-jetty3-1383861073911/solr.xml
[junit4] 2> 2843571 T8089 oasc.ConfigSolrXml.<init> Config-defined core root directory:
[junit4] 2> 2843571 T8089 oasc.CoreContainer.<init> New CoreContainer 1923603230
[junit4] 2> 2843572 T8089 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.DeleteReplicaTest-jetty3-1383861073911/]
[junit4] 2> 2843573 T8089 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 2843574 T8089 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4] 2> 2843575 T8089 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 2843575 T8089 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 2843575 T8089 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 2843575 T8089 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 2843576 T8089 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 2843576 T8089 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 2843576 T8089 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 2843583 T8089 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 2843584 T8089 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 2843584 T8089 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:51527/solr
[junit4] 2> 2843584 T8089 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 2843585 T8089 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 2843593 T8155 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@30954f94 name:ZooKeeperConnection Watcher:127.0.0.1:51527 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 2843594 T8089 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 2843602 T8089 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 2843607 T8157 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6660506f name:ZooKeeperConnection Watcher:127.0.0.1:51527/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 2843607 T8089 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 2843620 T8089 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 2844633 T8089 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:51540_n_ok
[junit4] 2> 2844638 T8089 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:51540_n_ok
[junit4] 2> 2844653 T8157 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 2844653 T8111 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 2844654 T8116 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 2844654 T8143 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 2844655 T8129 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 2844679 T8089 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.DeleteReplicaTest-jetty3-1383861073911
[junit4] 2> 2844808 T8089 oasc.CorePropertiesLocator.discoverUnder Found core conf in /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.DeleteReplicaTest-jetty3-1383861073911/conf/
[junit4] 2> 2844816 T8089 oasc.CorePropertiesLocator.discover Found 1 core definitions
[junit4] 2> 2844817 T8089 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0
[junit4] 2> 2844817 T8089 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 2845120 T8089 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 2845125 T8089 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:51543
[junit4] 2> 2845125 T8089 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 2845126 T8089 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 2845126 T8089 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.DeleteReplicaTest-jetty4-1383861075593
[junit4] 2> 2845126 T8089 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.DeleteReplicaTest-jetty4-1383861075593/'
[junit4] 2> 2845158 T8089 oasc.ConfigSolr.fromFile Loading container configuration from /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.DeleteReplicaTest-jetty4-1383861075593/solr.xml
[junit4] 2> 2845276 T8089 oasc.ConfigSolrXml.<init> Config-defined core root directory:
[junit4] 2> 2845277 T8089 oasc.CoreContainer.<init> New CoreContainer 1902554734
[junit4] 2> 2845277 T8089 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.DeleteReplicaTest-jetty4-1383861075593/]
[junit4] 2> 2845280 T8089 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 2845280 T8089 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4] 2> 2845281 T8089 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 2845281 T8089 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 2845282 T8089 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 2845282 T8089 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 2845283 T8089 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 2845284 T8089 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 2845285 T8089 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 2845291 T8089 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 2845291 T8089 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 2845292 T8089 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:51527/solr
[junit4] 2> 2845292 T8089 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 2845295 T8089 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 2845300 T8169 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@638d8000 name:ZooKeeperConnection Watcher:127.0.0.1:51527 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 2845300 T8089 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 2845306 T8089 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 2845311 T8171 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@49c9da5 name:ZooKeeperConnection Watcher:127.0.0.1:51527/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 2845311 T8089 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 2845323 T8089 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 2846333 T8089 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:51543_n_ok
[junit4] 2> 2846337 T8089 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:51543_n_ok
[junit4] 2> 2846345 T8171 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4] 2> 2846346 T8111 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4] 2> 2846346 T8143 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4] 2> 2846346 T8157 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4] 2> 2846347 T8116 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4] 2> 2846346 T8129 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4] 2> 2846361 T8089 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.DeleteReplicaTest-jetty4-1383861075593
[junit4] 2> 2846520 T8089 oasc.CorePropertiesLocator.discoverUnder Found core conf in /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.DeleteReplicaTest-jetty4-1383861075593/conf/
[junit4] 2> 2846528 T8089 oasc.CorePropertiesLocator.discover Found 1 core definitions
[junit4] 2> 2846529 T8089 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0
[junit4] 2> 2846529 T8089 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 2846535 T8089 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 2846543 T8174 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@72cc456e name:ZooKeeperConnection Watcher:127.0.0.1:51527/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 2846544 T8089 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 2846547 T8089 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 2846559 T8089 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 2846563 T8176 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@23fcffdb name:ZooKeeperConnection Watcher:127.0.0.1:51527/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 2846563 T8089 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 2846567 T8089 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 2846577 T8139 oasha.CollectionsHandler.handleCreateAction Creating Collection : numShards=2&maxShardsPerNode=2&name=delLiveColl&replicationFactor=2&action=CREATE&wt=javabin&version=2
[junit4] 2> 2846580 T8111 oasc.DistributedQueue$LatchChildWatcher.process Watcher fired on path: /overseer/collection-queue-work state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2846583 T8113 oasc.OverseerCollectionProcessor.run Overseer Collection Processor: Get the message id:/overseer/collection-queue-work/qn-0000000000 message:{
[junit4] 2> "operation":"createcollection",
[junit4] 2> "fromApi":"true",
[junit4] 2> "name":"delLiveColl",
[junit4] 2> "replicationFactor":"2",
[junit4] 2> "numShards":"2",
[junit4] 2> "maxShardsPerNode":"2"}
[junit4] 2> 2846583 T8113 oasc.OverseerCollectionProcessor.processMessage WARN OverseerCollectionProcessor.processMessage : createcollection , {
[junit4] 2> "operation":"createcollection",
[junit4] 2> "fromApi":"true",
[junit4] 2> "name":"delLiveColl",
[junit4] 2> "replicationFactor":"2",
[junit4] 2> "numShards":"2",
[junit4] 2> "maxShardsPerNode":"2"}
[junit4] 2> 2846798 T8112 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 2846801 T8112 oasc.Overseer$ClusterStateUpdater.buildCollection building a new collection: delLiveColl
[junit4] 2> 2846801 T8112 oasc.Overseer$ClusterStateUpdater.createCollection Create collection delLiveColl with shards [shard1, shard2]
[junit4] 2> 2846816 T8176 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 2846817 T8174 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 2846817 T8129 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 2846818 T8143 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 2846818 T8116 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 2846818 T8111 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 2846819 T8157 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 2846820 T8171 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 2846895 T8113 oasc.OverseerCollectionProcessor.createCollection going to create cores replicas shardNames [shard1, shard2] , repFactor : 2
[junit4] 2> 2846895 T8113 oasc.OverseerCollectionProcessor.createCollection Creating shard delLiveColl_shard1_replica1 as part of slice shard1 of collection delLiveColl on 127.0.0.1:51537_n_ok
[junit4] 2> 2846896 T8113 oasc.OverseerCollectionProcessor.createCollection Creating shard delLiveColl_shard1_replica2 as part of slice shard1 of collection delLiveColl on 127.0.0.1:51543_n_ok
[junit4] 2> 2846896 T8113 oasc.OverseerCollectionProcessor.createCollection Creating shard delLiveColl_shard2_replica1 as part of slice shard2 of collection delLiveColl on 127.0.0.1:51530_n_ok
[junit4] 2> 2846900 T8113 oasc.OverseerCollectionProcessor.createCollection Creating shard delLiveColl_shard2_replica2 as part of slice shard2 of collection delLiveColl on 127.0.0.1:51540_n_ok
[junit4] 2> 2846904 T8166 oasc.ZkController.publish publishing core=delLiveColl_shard1_replica2 state=down
[junit4] 2> 2846904 T8135 oasc.ZkController.publish publishing core=delLiveColl_shard1_replica1 state=down
[junit4] 2> 2846909 T8105 oasc.ZkController.publish publishing core=delLiveColl_shard2_replica1 state=down
[junit4] 2> 2846909 T8150 oasc.ZkController.publish publishing core=delLiveColl_shard2_replica2 state=down
[junit4] 2> 2846910 T8166 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 2846911 T8135 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 2846912 T8105 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 2846912 T8150 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 2848340 T8112 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 2848342 T8112 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:51543/n_ok",
[junit4] 2> "core":"delLiveColl_shard1_replica2",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:51543_n_ok",
[junit4] 2> "shard":"shard1",
[junit4] 2> "shard_range":null,
[junit4] 2> "shard_state":"active",
[junit4] 2> "shard_parent":null,
[junit4] 2> "collection":"delLiveColl",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":null}
[junit4] 2> 2848358 T8112 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:51537/n_ok",
[junit4] 2> "core":"delLiveColl_shard1_replica1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:51537_n_ok",
[junit4] 2> "shard":"shard1",
[junit4] 2> "shard_range":null,
[junit4] 2> "shard_state":"active",
[junit4] 2> "shard_parent":null,
[junit4] 2> "collection":"delLiveColl",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":null}
[junit4] 2> 2848373 T8112 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:51530/n_ok",
[junit4] 2> "core":"delLiveColl_shard2_replica1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:51530_n_ok",
[junit4] 2> "shard":"shard2",
[junit4] 2> "shard_range":null,
[junit4] 2> "shard_state":"active",
[junit4] 2> "shard_parent":null,
[junit4] 2> "collection":"delLiveColl",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":null}
[junit4] 2> 2848385 T8112 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:51540/n_ok",
[junit4] 2> "core":"delLiveColl_shard2_replica2",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:51540_n_ok",
[junit4] 2> "shard":"shard2",
[junit4] 2> "shard_range":null,
[junit4] 2> "shard_state":"active",
[junit4] 2> "shard_parent":null,
[junit4] 2> "collection":"delLiveColl",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":null}
[junit4] 2> 2848399 T8171 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 2848399 T8176 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 2848399 T8157 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 2848399 T8116 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 2848399 T8174 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 2848399 T8111 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 2848399 T8143 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 2848399 T8129 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 2848913 T8166 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for delLiveColl_shard1_replica2
[junit4] 2> 2848914 T8135 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for delLiveColl_shard1_replica1
[junit4] 2> 2848916 T8105 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for delLiveColl_shard2_replica1
[junit4] 2> 2848917 T8166 oasc.CoreContainer.create Creating SolrCore 'delLiveColl_shard1_replica2' using instanceDir: ../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.DeleteReplicaTest-jetty4-1383861075593/delLiveColl_shard1_replica2
[junit4] 2> 2848918 T8166 oasc.ZkController.createCollectionZkNode Check for collection zkNode:delLiveColl
[junit4] 2> 2848916 T8135 oasc.CoreContainer.create Creating SolrCore 'delLiveColl_shard1_replica1' using instanceDir: ./org.apache.solr.cloud.DeleteReplicaTest-jetty2-1383861072270/delLiveColl_shard1_replica1
[junit4] 2> 2848919 T8135 oasc.ZkController.createCollectionZkNode Check for collection zkNode:delLiveColl
[junit4] 2> 2848917 T8150 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for delLiveColl_shard2_replica2
[junit4] 2> 2848920 T8105 oasc.CoreContainer.create Creating SolrCore 'delLiveColl_shard2_replica1' using instanceDir: ../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.DeleteReplicaTest-controljetty-1383861069247/delLiveColl_shard2_replica1
[junit4] 2> 2848921 T8105 oasc.ZkController.createCollectionZkNode Check for collection zkNode:delLiveColl
[junit4] 2> 2848923 T8150 oasc.CoreContainer.create Creating SolrCore 'delLiveColl_shard2_replica2' using instanceDir: ../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.DeleteReplicaTest-jetty3-1383861073911/delLiveColl_shard2_replica2
[junit4] 2> 2848923 T8150 oasc.ZkController.createCollectionZkNode Check for collection zkNode:delLiveColl
[junit4] 2> 2848924 T8166 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:delLiveColl
[junit4] 2> 2848924 T8166 oasc.ZkController.getConfName Looking for collection configName
[junit4] 2> 2848925 T8135 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:delLiveColl
[junit4] 2> 2848926 T8135 oasc.ZkController.getConfName Looking for collection configName
[junit4] 2> 2848924 T8105 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:delLiveColl
[junit4] 2> 2848926 T8105 oasc.ZkController.getConfName Looking for collection configName
[junit4] 2> 2848929 T8150 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:delLiveColl
[junit4] 2> 2848930 T8150 oasc.ZkController.getConfName Looking for collection configName
[junit4] 2> 2848932 T8166 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
[junit4] 2> 2848933 T8166 oascc.SolrZkClient.makePath makePath: /collections/delLiveColl
[junit4] 2> 2848933 T8105 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
[junit4] 2> 2848934 T8150 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
[junit4] 2> 2848935 T8150 oascc.SolrZkClient.makePath makePath: /collections/delLiveColl
[junit4] 2> 2848933 T8135 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
[junit4] 2> 2848937 T8135 oascc.SolrZkClient.makePath makePath: /collections/delLiveColl
[junit4] 2> 2848934 T8105 oascc.SolrZkClient.makePath makePath: /collections/delLiveColl
[junit4] 2> 2848943 T8166 oasc.ZkController.readConfigName Load collection config from:/collections/delLiveColl
[junit4] 2> 2848944 T8150 oasc.ZkController.readConfigName Load collection config from:/collections/delLiveColl
[junit4] 2> 2848947 T8105 oasc.ZkController.readConfigName Load collection config from:/collections/delLiveColl
[junit4] 2> 2848948 T8135 oasc.ZkController.readConfigName Load collection config from:/collections/delLiveColl
[junit4] 2> 2848949 T8150 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.DeleteReplicaTest-jetty3-1383861073911/delLiveColl_shard2_replica2/'
[junit4] 2> 2848949 T8166 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.DeleteReplicaTest-jetty4-1383861075593/delLiveColl_shard1_replica2/'
[junit4] 2> 2848954 T8135 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.DeleteReplicaTest-jetty2-1383861072270/delLiveColl_shard1_replica1/'
[junit4] 2> 2848956 T8105 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.DeleteReplicaTest-controljetty-1383861069247/delLiveColl_shard2_replica1/'
[junit4] 2> 2849170 T8150 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4] 2> 2849184 T8166 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4] 2> 2849202 T8135 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4] 2> 2849204 T8105 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4] 2> 2849382 T8166 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 2849393 T8150 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 2849394 T8105 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 2849397 T8150 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 2849395 T8135 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 2849400 T8135 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 2849394 T8166 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 2849421 T8105 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 2849432 T8150 oass.IndexSchema.readSchema [delLiveColl_shard2_replica2] Schema name=test
[junit4] 2> 2849441 T8166 oass.IndexSchema.readSchema [delLiveColl_shard1_replica2] Schema name=test
[junit4] 2> 2849439 T8135 oass.IndexSchema.readSchema [delLiveColl_shard1_replica1] Schema name=test
[junit4] 2> 2849451 T8105 oass.IndexSchema.readSchema [delLiveColl_shard2_replica1] Schema name=test
[junit4] 2> 2851129 T8166 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 2851132 T8135 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 2851155 T8166 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 2851159 T8166 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 2851167 T8135 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 2851172 T8135 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 2851177 T8105 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 2851187 T8105 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 2851190 T8105 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 2851198 T8150 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 2851213 T8150 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 2851221 T8150 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 2851233 T8166 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 2851234 T8135 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 2851244 T8166 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 2851244 T8135 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 2851244 T8105 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 2851250 T8135 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 2851253 T8166 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 2851255 T8135 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 2851256 T8166 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 2851257 T8166 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 2851258 T8166 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 2851256 T8135 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 2851259 T8135 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 2851260 T8105 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 2851262 T8135 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 2851262 T8135 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 2851263 T8135 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 2851263 T8135 oasc.SolrCore.<init> [delLiveColl_shard1_replica1] Opening new SolrCore at ./org.apache.solr.cloud.DeleteReplicaTest-jetty2-1383861072270/delLiveColl_shard1_replica1/, dataDir=./org.apache.solr.cloud.DeleteReplicaTest-jetty2-1383861072270/delLiveColl_shard1_replica1/data/
[junit4] 2> 2851264 T8135 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@3a9906f2
[junit4] 2> 2851265 T8135 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.DeleteReplicaTest-jetty2-1383861072270/delLiveColl_shard1_replica1/data
[junit4] 2> 2851267 T8135 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.DeleteReplicaTest-jetty2-1383861072270/delLiveColl_shard1_replica1/data/index/
[junit4] 2> 2851268 T8135 oasc.SolrCore.initIndex WARN [delLiveColl_shard1_replica1] Solr index directory './org.apache.solr.cloud.DeleteReplicaTest-jetty2-1383861072270/delLiveColl_shard1_replica1/data/index' doesn't exist. Creating new index...
[junit4] 2> 2851268 T8135 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.DeleteReplicaTest-jetty2-1383861072270/delLiveColl_shard1_replica1/data/index
[junit4] 2> 2851269 T8135 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=26, maxMergeAtOnceExplicit=22, maxMergedSegmentMB=67.2294921875, floorSegmentMB=0.3037109375, forceMergeDeletesPctAllowed=24.404266042911825, segmentsPerTier=46.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
[junit4] 2> 2851270 T8166 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 2851271 T8166 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 2851271 T8105 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 2851271 T8135 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@3c711369 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@229e89d8),segFN=segments_1,generation=1}
[junit4] 2> 2851272 T8135 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 2851272 T8166 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 2851274 T8166 oasc.SolrCore.<init> [delLiveColl_shard1_replica2] Opening new SolrCore at ../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.DeleteReplicaTest-jetty4-1383861075593/delLiveColl_shard1_replica2/, dataDir=../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.DeleteReplicaTest-jetty4-1383861075593/delLiveColl_shard1_replica2/data/
[junit4] 2> 2851275 T8166 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@3a9906f2
[junit4] 2> 2851275 T8166 oasc.CachingDirectoryFactory.get return new directory for ../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.DeleteReplicaTest-jetty4-1383861075593/delLiveColl_shard1_replica2/data
[junit4] 2> 2851276 T8166 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.DeleteReplicaTest-jetty4-1383861075593/delLiveColl_shard1_replica2/data/index/
[junit4] 2> 2851276 T8166 oasc.SolrCore.initIndex WARN [delLiveColl_shard1_replica2] Solr index directory '../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.DeleteReplicaTest-jetty4-1383861075593/delLiveColl_shard1_replica2/data/index' doesn't exist. Creating new index...
[junit4] 2> 2851277 T8166 oasc.CachingDirectoryFactory.get return new directory for ../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.DeleteReplicaTest-jetty4-1383861075593/delLiveColl_shard1_replica2/data/index
[junit4] 2> 2851277 T8166 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=26, maxMergeAtOnceExplicit=22, maxMergedSegmentMB=67.2294921875, floorSegmentMB=0.3037109375, forceMergeDeletesPctAllowed=24.404266042911825, segmentsPerTier=46.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
[junit4] 2> 2851279 T8105 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 2851282 T8105 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 2851280 T8135 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 2851283 T8105 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 2851280 T8166 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@3a929669 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@34a1087b),segFN=segments_1,generation=1}
[junit4] 2> 2851284 T8135 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 2851286 T8135 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 2851287 T8135 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 2851285 T8166 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 2851288 T8150 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 2851287 T8135 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 2851295 T8166 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 2851287 T8105 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 2851296 T8166 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 2851296 T8105 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 2851297 T8166 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 2851295 T8135 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 2851294 T8150 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 2851298 T8135 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 2851298 T8166 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 2851300 T8166 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 2851301 T8166 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 2851302 T8166 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 2851302 T8166 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 2851302 T8166 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 2851303 T8166 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 2851304 T8166 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 2851304 T8166 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 2851304 T8166 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 2851305 T8166 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 2851306 T8166 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 2851307 T8150 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 2851297 T8105 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 2851309 T8105 oasc.SolrCore.<init> [delLiveColl_shard2_replica1] Opening new SolrCore at ../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.DeleteReplicaTest-controljetty-1383861069247/delLiveColl_shard2_replica1/, dataDir=../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.DeleteReplicaTest-controljetty-1383861069247/delLiveColl_shard2_replica1/data/
[junit4] 2> 2851309 T8105 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@3a9906f2
[junit4] 2> 2851310 T8105 oasc.CachingDirectoryFactory.get return new directory for ../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.DeleteReplicaTest-controljetty-1383861069247/delLiveColl_shard2_replica1/data
[junit4] 2> 2851300 T8135 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 2851311 T8135 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 2851313 T8135 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 2851313 T8135 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 2851314 T8135 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 2851314 T8135 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 2851314 T8135 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 2851315 T8135 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 2851311 T8105 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.DeleteReplicaTest-controljetty-1383861069247/delLiveColl_shard2_replica1/data/index/
[junit4] 2> 2851317 T8105 oasc.SolrCore.initIndex WARN [delLiveColl_shard2_replica1] Solr index directory '../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.DeleteReplicaTest-controljetty-1383861069247/delLiveColl_shard2_replica1/data/index' doesn't exist. Creating new index...
[junit4] 2> 2851318 T8105 oasc.CachingDirectoryFactory.get return new directory for ../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.DeleteReplicaTest-controljetty-1383861069247/delLiveColl_shard2_replica1/data/index
[junit4] 2> 2851318 T8105 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=26, maxMergeAtOnceExplicit=22, maxMergedSegmentMB=67.2294921875, floorSegmentMB=0.3037109375, forceMergeDeletesPctAllowed=24.404266042911825, segmentsPerTier=46.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
[junit4] 2> 2851312 T8150 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 2851319 T8150 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 2851320 T8150 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 2851321 T8105 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@1aff73e8 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@73a77442),segFN=segments_1,generation=1}
[junit4] 2> 2851321 T8105 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 2851323 T8166 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 2851325 T8105 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 2851326 T8105 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 2851326 T8105 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 2851327 T8105 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 2851327 T8105 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 2851327 T8105 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 2851329 T8105 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 2851331 T8135 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 2851331 T8105 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 2851331 T8105 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 2851332 T8105 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 2851333 T8105 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 2851333 T8105 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 2851333 T8105 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 2851334 T8105 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 2851334 T8166 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 2851335 T8166 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 2851336 T8166 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=45, maxMergeAtOnceExplicit=16, maxMergedSegmentMB=26.6845703125, floorSegmentMB=0.2099609375, forceMergeDeletesPctAllowed=16.574541677987533, segmentsPerTier=28.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.38304708617791483
[junit4] 2> 2851335 T8105 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 2851335 T8150 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 2851338 T8150 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 2851339 T8150 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 2851340 T8166 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@3a929669 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@34a1087b),segFN=segments_1,generation=1}
[junit4] 2> 2851342 T8166 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 2851339 T8105 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 2851342 T8135 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 2851343 T8135 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 2851342 T8166 oass.SolrIndexSearcher.<init> Opening Searcher@476ffef2 main
[junit4] 2> 2851341 T8150 oasc.SolrCore.<init> [delLiveColl_shard2_replica2] Opening new SolrCore at ../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.DeleteReplicaTest-jetty3-1383861073911/delLiveColl_shard2_replica2/, dataDir=../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.DeleteReplicaTest-jetty3-1383861073911/delLiveColl_shard2_replica2/data/
[junit4] 2> 2851345 T8150 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@3a9906f2
[junit4] 2> 2851347 T8150 oasc.CachingDirectoryFactory.get return new directory for ../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.DeleteReplicaTest-jetty3-1383861073911/delLiveColl_shard2_replica2/data
[junit4] 2> 2851348 T8150 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.DeleteReplicaTest-jetty3-1383861073911/delLiveColl_shard2_replica2/data/index/
[junit4] 2> 2851348 T8150 oasc.SolrCore.initIndex WARN [delLiveColl_shard2_replica2] Solr index directory '../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.DeleteReplicaTest-jetty3-1383861073911/delLiveColl_shard2_replica2/data/index' doesn't exist. Creating new index...
[junit4] 2> 2851349 T8150 oasc.CachingDirectoryFactory.get return new directory for ../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.DeleteReplicaTest-jetty3-1383861073911/delLiveColl_shard2_replica2/data/index
[junit4] 2> 2851350 T8150 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=26, maxMergeAtOnceExplicit=22, maxMergedSegmentMB=67.2294921875, floorSegmentMB=0.3037109375, forceMergeDeletesPctAllowed=24.404266042911825, segmentsPerTier=46.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
[junit4] 2> 2851350 T8105 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 2851351 T8105 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 2851348 T8135 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=45, maxMergeAtOnceExplicit=16, maxMergedSegmentMB=26.6845703125, floorSegmentMB=0.2099609375, forceMergeDeletesPctAllowed=16.574541677987533, segmentsPerTier=28.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.38304708617791483
[junit4] 2> 2851352 T8150 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@376dac9 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@130b9d12),segFN=segments_1,generation=1}
[junit4] 2> 2851353 T8150 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 2851351 T8105 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=45, maxMergeAtOnceExplicit=16, maxMergedSegmentMB=26.6845703125, floorSegmentMB=0.2099609375, forceMergeDeletesPctAllowed=16.574541677987533, segmentsPerTier=28.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.38304708617791483
[junit4] 2> 2851421 T8105 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@1aff73e8 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@73a77442),segFN=segments_1,generation=1}
[junit4] 2> 2851422 T8105 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 2851422 T8105 oass.SolrIndexSearcher.<init> Opening Searcher@548488bd main
[junit4] 2> 2851421 T8150 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 2851425 T8150 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 2851425 T8150 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 2851426 T8150 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "sto
[...truncated too long message...]
ctoryTest, TestCSVLoader, PathHierarchyTokenizerFactoryTest, DistributedSpellCheckComponentTest, SliceStateUpdateTest, BasicZkTest, TermVectorComponentDistributedTest, CoreContainerCoreInitFailuresTest, WordBreakSolrSpellCheckerTest, TestAnalyzedSuggestions, DocValuesMultiTest, CSVRequestHandlerTest, TestMultiCoreConfBootstrap, BasicFunctionalityTest, FileUtilsTest, CustomCollectionTest, TestQueryUtils, XmlUpdateRequestHandlerTest, TestRecovery, CopyFieldTest, TestFieldTypeCollectionResource, TestTrie, TestFieldResource, TestRandomFaceting, SystemInfoHandlerTest, SpellCheckComponentTest, CoreAdminHandlerTest, TestDFRSimilarityFactory, CurrencyFieldXmlFileTest, RecoveryZkTest, SimpleFacetsTest, TestSchemaResource, TestFastLRUCache, TestCodecSupport, InfoHandlerTest, AutoCommitTest, TestCopyFieldCollectionResource, TestSolrQueryParser, HdfsRecoveryZkTest, TestRandomMergePolicy, TermVectorComponentTest, ChaosMonkeySafeLeaderTest, HdfsSyncSliceTest, LeaderElectionIntegrationTest, BlockDirectoryTest, CollectionsAPIDistributedZkTest, SpellPossibilityIteratorTest, TestInfoStreamLogging, TestNoOpRegenerator, HardAutoCommitTest, RegexBoostProcessorTest, AddBlockUpdateTest, TestDocumentBuilder, TestZkChroot, TestAnalyzeInfixSuggestions, FileBasedSpellCheckerTest, TestWriterPerf, PingRequestHandlerTest, TestRealTimeGet, TestJoin, SortByFunctionTest, TestPseudoReturnFields, TestRandomDVFaceting, LukeRequestHandlerTest, BasicDistributedZkTest, ZkSolrClientTest, PluginInfoTest, JsonLoaderTest, NumericFieldsTest, StressHdfsTest, TestSolrXmlPersistor, ZkNodePropsTest, TestLMJelinekMercerSimilarityFactory, ZkControllerTest, SOLR749Test, TestStressVersions, TestManagedSchemaFieldResource, TestPHPSerializedResponseWriter, TestIndexSearcher, TestUniqueKeyFieldResource, AssignTest, DocumentBuilderTest, SolrRequestParserTest, TestStressLucene, TestIBSimilarityFactory, TestFieldCollectionResource, SpatialFilterTest, TestFunctionQuery, DateFieldTest, OverseerTest, DeleteInactiveReplicaTest, CachingDirectoryFactoryTest, TestManagedSchema, TestSolrCoreProperties, BlockCacheTest, TestDefaultSearchFieldResource, TestSolrDeletionPolicy1, ShardSplitTest, QueryParsingTest, SolrTestCaseJ4Test, SuggesterTSTTest, TestHashPartitioner, SearchHandlerTest, HdfsBasicDistributedZk2Test, HdfsDirectoryTest, TestNRTOpen, DirectUpdateHandlerOptimizeTest, FieldMutatingUpdateProcessorTest, TestSearchPerf, AddSchemaFieldsUpdateProcessorFactoryTest, StandardRequestHandlerTest, TestXIncludeConfig, OpenExchangeRatesOrgProviderTest, TestCloudManagedSchema, TestFaceting, FieldAnalysisRequestHandlerTest, TestMergePolicyConfig, DefaultValueUpdateProcessorTest, TestSolrXmlPersistence, UnloadDistributedZkTest, TestJmxMonitoredMap, TestLFUCache, DisMaxRequestHandlerTest, TestCoreContainer, TestArbitraryIndexDir, TestRemoteStreaming, CoreAdminCreateDiscoverTest, PeerSyncTest, OpenCloseCoreStressTest, BasicDistributedZk2Test, SyncSliceTest, ShardRoutingTest, FullSolrCloudDistribCmdsTest, TestReplicationHandler, ClusterStateUpdateTest, ZkCLITest, ShardRoutingCustomTest, TestDistributedSearch, TestDistributedGrouping, TestStressReorder, DistributedTermsComponentTest, TestRangeQuery, StatsComponentTest, SolrCmdDistributorTest, QueryElevationComponentTest, BadIndexSchemaTest, ConvertedLegacyTest, TestSort, TestFiltering, TestLazyCores, DirectUpdateHandlerTest, TestBadConfig, OverseerCollectionProcessorTest, SoftAutoCommitTest, ShowFileRequestHandlerTest, CurrencyFieldOpenExchangeTest, DistributedQueryElevationComponentTest, SolrIndexSplitterTest, SimplePostToolTest, TestCoreDiscovery, AnalysisAfterCoreReloadTest, SignatureUpdateProcessorFactoryTest, SpellCheckCollatorTest, TestFoldingMultitermQuery, DocValuesTest, SuggesterTest, SuggesterWFSTTest, NoCacheHeaderTest, SchemaVersionSpecificBehaviorTest, SolrCoreCheckLockOnStartupTest, TestUpdate, TestAtomicUpdateErrorCases, QueryEqualityTest, StatelessScriptUpdateProcessorFactoryTest, DistanceFunctionTest, SolrInfoMBeanTest, XsltUpdateRequestHandlerTest, DebugComponentTest, CacheHeaderTest, IndexBasedSpellCheckerTest, TestSurroundQueryParser, DirectSolrSpellCheckerTest, TestReversedWildcardFilterFactory, DocumentAnalysisRequestHandlerTest, PrimitiveFieldTypeTest, TestOmitPositions, TermsComponentTest, RequestHandlersTest, TestValueSourceCache, MoreLikeThisHandlerTest, RequiredFieldsTest, FastVectorHighlighterTest, IndexSchemaRuntimeFieldTest, LoggingHandlerTest, SolrPluginUtilsTest, TestJmxIntegration, TestCollationField, ReturnFieldsTest, UpdateRequestProcessorFactoryTest, JSONWriterTest, TestCSVResponseWriter, MBeansHandlerTest, BinaryUpdateRequestHandlerTest, TestPartialUpdateDeduplication, TestComponentsName, HighlighterConfigTest, TestQuerySenderListener, UpdateParamsTest, AlternateDirectoryTest, TestSolrIndexConfig, TestQuerySenderNoQuery, SolrIndexConfigTest, TestStressRecovery, TestSolrDeletionPolicy2, MultiTermTest, SampleTest, TestDocSet, TestBinaryField, MinimalSchemaTest, TestElisionMultitermQuery, OutputWriterTest, TestPostingsSolrHighlighter, DirectSolrConnectionTest, NotRequiredUniqueKeyTest, TestLuceneMatchVersion, TestPhraseSuggestions, SynonymTokenizerTest, EchoParamsTest, TestSweetSpotSimilarityFactory, TestPerFieldSimilarity, TestNumberUtils, TestLMDirichletSimilarityFactory, TestDefaultSimilarityFactory, TestFastWriter, LegacyHTMLStripCharFilterTest, TimeZoneUtilsTest, TestFastOutputStream, ScriptEngineTest, URLClassifyProcessorTest, DateMathParserTest, TestSolrXMLSerializer, PreAnalyzedFieldTest, TestSystemIdResolver, PrimUtilsTest, TestSuggestSpellingConverter, SpellingQueryConverterTest, DOMUtilTest, RAMDirectoryFactoryTest, ClusterStateTest, TestLRUCache, TestUtils, UUIDFieldTest, CircularListTest, AliasIntegrationTest, DeleteReplicaTest]
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=DeleteReplicaTest -Dtests.seed=70519D0640F75D32 -Dtests.slow=true -Dtests.locale=ar_AE -Dtests.timezone=America/Santa_Isabel -Dtests.file.encoding=US-ASCII
[junit4] ERROR 0.00s | DeleteReplicaTest (suite) <<<
[junit4] > Throwable #1: java.lang.AssertionError: ERROR: SolrZkClient opens=15 closes=14
[junit4] > at __randomizedtesting.SeedInfo.seed([70519D0640F75D32]:0)
[junit4] > at org.apache.solr.SolrTestCaseJ4.endTrackingZkClients(SolrTestCaseJ4.java:343)
[junit4] > at org.apache.solr.SolrTestCaseJ4.afterClass(SolrTestCaseJ4.java:135)
[junit4] > at java.lang.Thread.run(Thread.java:744)Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: 2 threads leaked from SUITE scope at org.apache.solr.cloud.DeleteReplicaTest:
[junit4] > 1) Thread[id=8176, name=TEST-DeleteReplicaTest.testDistribSearch-seed#[70519D0640F75D32]-EventThread, state=WAITING, group=TGRP-DeleteReplicaTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
[junit4] > at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4] > at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:491)
[junit4] > 2) Thread[id=8175, name=TEST-DeleteReplicaTest.testDistribSearch-seed#[70519D0640F75D32]-SendThread(localhost:51527), state=TIMED_WAITING, group=TGRP-DeleteReplicaTest]
[junit4] > at java.lang.Thread.sleep(Native Method)
[junit4] > at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:86)
[junit4] > at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:937)
[junit4] > at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:993)
[junit4] > at __randomizedtesting.SeedInfo.seed([70519D0640F75D32]:0)Throwable #3: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
[junit4] > 1) Thread[id=8175, name=TEST-DeleteReplicaTest.testDistribSearch-seed#[70519D0640F75D32]-SendThread(localhost:51527), state=TIMED_WAITING, group=TGRP-DeleteReplicaTest]
[junit4] > at java.lang.Thread.sleep(Native Method)
[junit4] > at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:86)
[junit4] > at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:937)
[junit4] > at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:993)
[junit4] > at __randomizedtesting.SeedInfo.seed([70519D0640F75D32]:0)
[junit4] Completed in 131.65s, 1 test, 1 failure, 3 errors <<< FAILURES!
[...truncated 115 lines...]
BUILD FAILED
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/build.xml:421: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/build.xml:401: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/build.xml:39: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build.xml:189: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/common-build.xml:471: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/lucene/common-build.xml:1271: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/lucene/common-build.xml:907: There were test failures: 334 suites, 1470 tests, 3 suite-level errors, 1 error, 200 ignored (5 assumptions)
Total time: 81 minutes 36 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 64bit/jdk1.7.0 -XX:+UseCompressedOops -XX:+UseG1GC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure