You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2014/08/07 19:43:05 UTC

[JENKINS] Lucene-Solr-NightlyTests-trunk - Build # 599 - Still Failing

Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-trunk/599/

3 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest

Error Message:
ERROR: SolrIndexSearcher opens=16 closes=15

Stack Trace:
java.lang.AssertionError: ERROR: SolrIndexSearcher opens=16 closes=15
	at __randomizedtesting.SeedInfo.seed([A07445A293ED20EE]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.solr.SolrTestCaseJ4.endTrackingSearchers(SolrTestCaseJ4.java:436)
	at org.apache.solr.SolrTestCaseJ4.afterClass(SolrTestCaseJ4.java:183)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:790)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest

Error Message:
2 threads leaked from SUITE scope at org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest:     1) Thread[id=197, name=qtp1182665876-197, state=WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]         at java.lang.Object.wait(Native Method)         at java.lang.Object.wait(Object.java:503)         at org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrServer.blockUntilFinished(ConcurrentUpdateSolrServer.java:374)         at org.apache.solr.update.StreamingSolrServers.blockUntilFinished(StreamingSolrServers.java:103)         at org.apache.solr.update.SolrCmdDistributor.blockAndDoRetries(SolrCmdDistributor.java:228)         at org.apache.solr.update.SolrCmdDistributor.finish(SolrCmdDistributor.java:89)         at org.apache.solr.update.processor.DistributedUpdateProcessor.doFinish(DistributedUpdateProcessor.java:766)         at org.apache.solr.update.processor.DistributedUpdateProcessor.finish(DistributedUpdateProcessor.java:1662)         at org.apache.solr.update.processor.LogUpdateProcessor.finish(LogUpdateProcessorFactory.java:179)         at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:83)         at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)         at org.apache.solr.core.SolrCore.execute(SolrCore.java:1984)         at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:780)         at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:420)         at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:208)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)         at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:137)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)         at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455)         at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:229)         at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)         at org.eclipse.jetty.server.handler.GzipHandler.handle(GzipHandler.java:301)         at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1077)         at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384)         at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)         at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009)         at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)         at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)         at org.eclipse.jetty.server.Server.handle(Server.java:368)         at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489)         at org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53)         at org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:953)         at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:1014)         at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:953)         at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:240)         at org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72)         at org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:745)    2) Thread[id=205, name=searcherExecutor-87-thread-1, state=WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:745)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 2 threads leaked from SUITE scope at org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest: 
   1) Thread[id=197, name=qtp1182665876-197, state=WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]
        at java.lang.Object.wait(Native Method)
        at java.lang.Object.wait(Object.java:503)
        at org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrServer.blockUntilFinished(ConcurrentUpdateSolrServer.java:374)
        at org.apache.solr.update.StreamingSolrServers.blockUntilFinished(StreamingSolrServers.java:103)
        at org.apache.solr.update.SolrCmdDistributor.blockAndDoRetries(SolrCmdDistributor.java:228)
        at org.apache.solr.update.SolrCmdDistributor.finish(SolrCmdDistributor.java:89)
        at org.apache.solr.update.processor.DistributedUpdateProcessor.doFinish(DistributedUpdateProcessor.java:766)
        at org.apache.solr.update.processor.DistributedUpdateProcessor.finish(DistributedUpdateProcessor.java:1662)
        at org.apache.solr.update.processor.LogUpdateProcessor.finish(LogUpdateProcessorFactory.java:179)
        at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:83)
        at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
        at org.apache.solr.core.SolrCore.execute(SolrCore.java:1984)
        at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:780)
        at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:420)
        at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:208)
        at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
        at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:137)
        at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
        at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455)
        at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:229)
        at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
        at org.eclipse.jetty.server.handler.GzipHandler.handle(GzipHandler.java:301)
        at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1077)
        at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384)
        at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
        at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009)
        at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
        at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
        at org.eclipse.jetty.server.Server.handle(Server.java:368)
        at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489)
        at org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53)
        at org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:953)
        at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:1014)
        at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:953)
        at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:240)
        at org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72)
        at org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:745)
   2) Thread[id=205, name=searcherExecutor-87-thread-1, state=WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
	at __randomizedtesting.SeedInfo.seed([A07445A293ED20EE]:0)


FAILED:  org.apache.solr.cloud.MultiThreadedOCPTest.testDistribSearch

Error Message:
We have a failed SPLITSHARD task

Stack Trace:
java.lang.AssertionError: We have a failed SPLITSHARD task
	at __randomizedtesting.SeedInfo.seed([A07445A293ED20EE:2192CBBAE4B240D2]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.apache.solr.cloud.MultiThreadedOCPTest.testTaskExclusivity(MultiThreadedOCPTest.java:125)
	at org.apache.solr.cloud.MultiThreadedOCPTest.doTest(MultiThreadedOCPTest.java:71)
	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:865)
	at sun.reflect.GeneratedMethodAccessor50.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 13326 lines...]
   [junit4] Suite: org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest
   [junit4]   2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-A07445A293ED20EE-001/init-core-data-001
   [junit4]   2> 64454 T154 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /pb_/
   [junit4]   2> 64489 T154 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
   [junit4]   2> 64490 T154 oasc.ZkTestServer.run STARTING ZK TEST SERVER
   [junit4]   1> client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 64491 T155 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
   [junit4]   2> 64590 T154 oasc.ZkTestServer.run start zk server on port:40090
   [junit4]   2> 64591 T154 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 64595 T161 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@517f37f5 name:ZooKeeperConnection Watcher:127.0.0.1:40090 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 64595 T154 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 64596 T154 oascc.SolrZkClient.makePath makePath: /solr
   [junit4]   2> 64600 T154 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 64601 T163 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@289f65df name:ZooKeeperConnection Watcher:127.0.0.1:40090/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 64602 T154 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 64602 T154 oascc.SolrZkClient.makePath makePath: /collections/collection1
   [junit4]   2> 64606 T154 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
   [junit4]   2> 64608 T154 oascc.SolrZkClient.makePath makePath: /collections/control_collection
   [junit4]   2> 64611 T154 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
   [junit4]   2> 64614 T154 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 64615 T154 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 64619 T154 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
   [junit4]   2> 64620 T154 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
   [junit4]   2> 64624 T154 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 64624 T154 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 64627 T154 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 64628 T154 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 64631 T154 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 64632 T154 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
   [junit4]   2> 64635 T154 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 64636 T154 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
   [junit4]   2> 64639 T154 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 64640 T154 oascc.SolrZkClient.makePath makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 64643 T154 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 64643 T154 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 64646 T154 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 64647 T154 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 64650 T154 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 64651 T154 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 64654 T154 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 64654 T154 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 67611 T154 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 67616 T154 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:53547
   [junit4]   2> 67617 T154 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 67617 T154 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 67617 T154 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-A07445A293ED20EE-001/tempDir-002
   [junit4]   2> 67618 T154 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-A07445A293ED20EE-001/tempDir-002/'
   [junit4]   2> 67678 T154 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-A07445A293ED20EE-001/tempDir-002/solr.xml
   [junit4]   2> 67792 T154 oasc.CoreContainer.<init> New CoreContainer 1286591841
   [junit4]   2> 67792 T154 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-A07445A293ED20EE-001/tempDir-002/]
   [junit4]   2> 67794 T154 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 67795 T154 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 67795 T154 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 67795 T154 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 67796 T154 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 67796 T154 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 67796 T154 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 67797 T154 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 67797 T154 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 67798 T154 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
   [junit4]   2> 67798 T154 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 67799 T154 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 67799 T154 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 67800 T154 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:40090/solr
   [junit4]   2> 67800 T154 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 67801 T154 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 67803 T174 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@10c2e46 name:ZooKeeperConnection Watcher:127.0.0.1:40090 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 67803 T154 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 67806 T154 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 67808 T176 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@52aea950 name:ZooKeeperConnection Watcher:127.0.0.1:40090/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 67808 T154 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 67809 T154 oascc.SolrZkClient.makePath makePath: /overseer/queue
   [junit4]   2> 67812 T154 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
   [junit4]   2> 67815 T154 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
   [junit4]   2> 67818 T154 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
   [junit4]   2> 67820 T154 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
   [junit4]   2> 67823 T154 oascc.SolrZkClient.makePath makePath: /live_nodes
   [junit4]   2> 67825 T154 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:53547_pb_
   [junit4]   2> 67826 T154 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:53547_pb_
   [junit4]   2> 67828 T154 oascc.SolrZkClient.makePath makePath: /overseer_elect
   [junit4]   2> 67830 T154 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
   [junit4]   2> 67834 T154 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:53547_pb_
   [junit4]   2> 67834 T154 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
   [junit4]   2> 67836 T154 oasc.Overseer.start Overseer (id=92237202507694083-127.0.0.1:53547_pb_-n_0000000000) starting
   [junit4]   2> 67839 T154 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
   [junit4]   2> 67847 T178 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
   [junit4]   2> 67847 T154 oascc.SolrZkClient.makePath makePath: /clusterstate.json
   [junit4]   2> 67849 T154 oascc.SolrZkClient.makePath makePath: /aliases.json
   [junit4]   2> 67851 T154 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 67853 T177 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
   [junit4]   2> 67875 T179 oasc.ZkController.publish publishing core=collection1 state=down collection=control_collection
   [junit4]   2> 67875 T179 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 67877 T179 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 67877 T180 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 67883 T177 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 67883 T177 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:53547/pb_",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:53547_pb_",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "collection":"control_collection",
   [junit4]   2> 	  "numShards":"1",
   [junit4]   2> 	  "core_node_name":null}
   [junit4]   2> 67884 T177 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with shards [shard1]
   [junit4]   2> 67914 T177 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
   [junit4]   2> 67917 T180 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 67918 T180 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 68878 T179 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
   [junit4]   2> 68878 T179 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
   [junit4]   2> 68879 T179 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 68879 T179 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
   [junit4]   2> 68880 T179 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 68881 T179 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-A07445A293ED20EE-001/tempDir-002/collection1/'
   [junit4]   2> 68882 T179 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-A07445A293ED20EE-001/tempDir-002/collection1/lib/classes/' to classloader
   [junit4]   2> 68883 T179 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-A07445A293ED20EE-001/tempDir-002/collection1/lib/README' to classloader
   [junit4]   2> 68883 T179 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-A07445A293ED20EE-001/tempDir-002/collection1/lib/.svn/' to classloader
   [junit4]   2> 68937 T179 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_5_0
   [junit4]   2> 68980 T179 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 68981 T179 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 69001 T179 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 69414 T179 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 69417 T179 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 69420 T179 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 69426 T179 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 69471 T179 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 69471 T179 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
   [junit4]   2> 69542 T179 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-A07445A293ED20EE-001/tempDir-002/collection1/, dataDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-A07445A293ED20EE-001/tempDir-001/control/data/
   [junit4]   2> 69543 T179 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@54b83713
   [junit4]   2> 69545 T179 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-A07445A293ED20EE-001/tempDir-001/control/data
   [junit4]   2> 69547 T179 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-A07445A293ED20EE-001/tempDir-001/control/data/index/
   [junit4]   2> 69548 T179 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-A07445A293ED20EE-001/tempDir-001/control/data/index' doesn't exist. Creating new index...
   [junit4]   2> 69549 T179 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-A07445A293ED20EE-001/tempDir-001/control/data/index
   [junit4]   2> 69550 T179 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=50, maxMergeAtOnceExplicit=16, maxMergedSegmentMB=0.4306640625, floorSegmentMB=1.0390625, forceMergeDeletesPctAllowed=3.007953945489976, segmentsPerTier=41.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 69582 T179 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-A07445A293ED20EE-001/tempDir-001/control/data/index,segFN=segments_1,generation=1}
   [junit4]   2> 69583 T179 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 69590 T179 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 69591 T179 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 69591 T179 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 69591 T179 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 69592 T179 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 69592 T179 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 69592 T179 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 69593 T179 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 69593 T179 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 69594 T179 oasc.RequestHandlers.initHandlersFromConfig created /update: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 69594 T179 oasc.RequestHandlers.initHandlersFromConfig created /update/json: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 69595 T179 oasc.RequestHandlers.initHandlersFromConfig created /update/csv: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 69596 T179 oasc.RequestHandlers.initHandlersFromConfig created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 69596 T179 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 69597 T179 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 69597 T179 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 69597 T179 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 69598 T179 oasc.RequestHandlers.initHandlersFromConfig WARN Multiple requestHandler registered to the same name: /update ignoring: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 69598 T179 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 69599 T179 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 69599 T179 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
   [junit4]   2> 69600 T179 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
   [junit4]   2> 69619 T179 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 69622 T179 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 69624 T179 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 69626 T179 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 69628 T179 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 69637 T179 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 69638 T179 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 69639 T179 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=3, maxMergeAtOnceExplicit=6, maxMergedSegmentMB=56.107421875, floorSegmentMB=0.4912109375, forceMergeDeletesPctAllowed=26.23316267623028, segmentsPerTier=12.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.28139084101318934
   [junit4]   2> 69641 T179 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-A07445A293ED20EE-001/tempDir-001/control/data/index,segFN=segments_1,generation=1}
   [junit4]   2> 69642 T179 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 69642 T179 oass.SolrIndexSearcher.<init> Opening Searcher@36e6512c[collection1] main
   [junit4]   2> 69643 T179 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
   [junit4]   2> 69644 T179 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 69644 T179 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 69656 T179 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 69657 T179 oasr.RestManager.init Initializing RestManager with initArgs: {}
   [junit4]   2> 69657 T179 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 69658 T179 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 69659 T179 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 69660 T179 oascc.SolrZkClient.makePath makePath: /configs/conf1/_rest_managed.json
   [junit4]   2> 69663 T179 oasr.ManagedResourceStorage$ZooKeeperStorageIO$1.close Wrote 38 bytes to new znode /configs/conf1/_rest_managed.json
   [junit4]   2> 69664 T179 oasr.ManagedResourceStorage$JsonStorage.store Saved JSON object to path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 69664 T179 oasr.RestManager.init Initializing 0 registered ManagedResources
   [junit4]   2> 69668 T181 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@36e6512c[collection1] main{UninvertingDirectoryReader()}
   [junit4]   2> 69674 T179 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 69675 T184 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:53547/pb_ collection:control_collection shard:shard1
   [junit4]   2> 69676 T154 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0
   [junit4]   2> 69676 T154 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 69714 T184 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 69721 T184 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
   [junit4]   2> 69723 T180 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 69724 T184 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
   [junit4]   2> 69724 T184 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
   [junit4]   2> 69725 T177 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> ASYNC  NEW_CORE C49 name=collection1 org.apache.solr.core.SolrCore@3856a77c url=http://127.0.0.1:53547/pb_/collection1 node=127.0.0.1:53547_pb_ C49_STATE=coll:control_collection core:collection1 props:{state=down, base_url=http://127.0.0.1:53547/pb_, core=collection1, node_name=127.0.0.1:53547_pb_}
   [junit4]   2> 69732 T184 C49 P53547 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:53547/pb_/collection1/
   [junit4]   2> 69732 T184 C49 P53547 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
   [junit4]   2> 69733 T184 C49 P53547 oasc.SyncStrategy.syncToMe http://127.0.0.1:53547/pb_/collection1/ has no replicas
   [junit4]   2> 69733 T184 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:53547/pb_/collection1/ shard1
   [junit4]   2> 69734 T180 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 69735 T180 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 69747 T184 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 69753 T180 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 69755 T177 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 69758 T180 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 69760 T154 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 69761 T186 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7ae41e92 name:ZooKeeperConnection Watcher:127.0.0.1:40090/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 69762 T154 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 69763 T154 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 69769 T154 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 69861 T180 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 69861 T187 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 69906 T184 oasc.ZkController.register We are http://127.0.0.1:53547/pb_/collection1/ and leader is http://127.0.0.1:53547/pb_/collection1/
   [junit4]   2> 69907 T184 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:53547/pb_
   [junit4]   2> 69907 T184 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
   [junit4]   2> 69907 T184 oasc.ZkController.publish publishing core=collection1 state=active collection=control_collection
   [junit4]   2> 69907 T184 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 69910 T180 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 69910 T188 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 69910 T189 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 69910 T184 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 69912 T177 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 69913 T177 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"active",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:53547/pb_",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:53547_pb_",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "collection":"control_collection",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "core_node_name":"core_node1"}
   [junit4]   2> 69916 T189 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 70019 T189 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 70019 T187 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 71199 T154 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 1
   [junit4]   2> 71200 T154 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 71202 T154 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:45461
   [junit4]   2> 71203 T154 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 71203 T154 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 71203 T154 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-A07445A293ED20EE-001/tempDir-003
   [junit4]   2> 71204 T154 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-A07445A293ED20EE-001/tempDir-003/'
   [junit4]   2> 71236 T154 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-A07445A293ED20EE-001/tempDir-003/solr.xml
   [junit4]   2> 71299 T154 oasc.CoreContainer.<init> New CoreContainer 95952472
   [junit4]   2> 71300 T154 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-A07445A293ED20EE-001/tempDir-003/]
   [junit4]   2> 71301 T154 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 71301 T154 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 71302 T154 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 71302 T154 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 71302 T154 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 71302 T154 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 71303 T154 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 71303 T154 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 71303 T154 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 71304 T154 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
   [junit4]   2> 71304 T154 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 71304 T154 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 71304 T154 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 71305 T154 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:40090/solr
   [junit4]   2> 71305 T154 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 71306 T154 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 71308 T200 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7ad02656 name:ZooKeeperConnection Watcher:127.0.0.1:40090 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 71308 T154 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 71310 T154 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 71312 T202 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@d8b69db name:ZooKeeperConnection Watcher:127.0.0.1:40090/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 71312 T154 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 71319 T154 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 72323 T154 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:45461_pb_
   [junit4]   2> 72324 T154 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:45461_pb_
   [junit4]   2> 72342 T204 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
   [junit4]   2> 72342 T204 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 72343 T189 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 72343 T188 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 72343 T180 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 72343 T204 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 72345 T177 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 72346 T177 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:45461/pb_",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:45461_pb_",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "core_node_name":null}
   [junit4]   2> 72346 T177 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1, shard2]
   [junit4]   2> 72346 T177 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
   [junit4]   2> 72349 T180 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 72349 T203 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 72350 T187 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 72349 T180 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 73344 T204 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
   [junit4]   2> 73345 T204 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
   [junit4]   2> 73346 T204 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 73346 T204 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
   [junit4]   2> 73347 T204 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 73348 T204 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-A07445A293ED20EE-001/tempDir-003/collection1/'
   [junit4]   2> 73349 T204 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-A07445A293ED20EE-001/tempDir-003/collection1/lib/.svn/' to classloader
   [junit4]   2> 73350 T204 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-A07445A293ED20EE-001/tempDir-003/collection1/lib/classes/' to classloader
   [junit4]   2> 73351 T204 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-A07445A293ED20EE-001/tempDir-003/collection1/lib/README' to classloader
   [junit4]   2> 73425 T204 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_5_0
   [junit4]   2> 73479 T204 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 73481 T204 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 73511 T204 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 73935 T204 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 73939 T204 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 73942 T204 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 73948 T204 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 73994 T204 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection collection1
   [junit4]   2> 73995 T204 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
   [junit4]   2> 73995 T204 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-A07445A293ED20EE-001/tempDir-003/collection1/, dataDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-A07445A293ED20EE-001/tempDir-001/jetty1/
   [junit4]   2> 73995 T204 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@54b83713
   [junit4]   2> 73997 T204 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-A07445A293ED20EE-001/tempDir-001/jetty1
   [junit4]   2> 73998 T204 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-A07445A293ED20EE-001/tempDir-001/jetty1/index/
   [junit4]   2> 73999 T204 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-A07445A293ED20EE-001/tempDir-001/jetty1/index' doesn't exist. Creating new index...
   [junit4]   2> 74001 T204 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-A07445A293ED20EE-001/tempDir-001/jetty1/index
   [junit4]   2> 74002 T204 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=50, maxMergeAtOnceExplicit=16, maxMergedSegmentMB=0.4306640625, floorSegmentMB=1.0390625, forceMergeDeletesPctAllowed=3.007953945489976, segmentsPerTier=41.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 74037 T204 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-A07445A293ED20EE-001/tempDir-001/jetty1/index,segFN=segments_1,generation=1}
   [junit4]   2> 74038 T204 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 74048 T204 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 74048 T204 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 74049 T204 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 74049 T204 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 74049 T204 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 74050 T204 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 74050 T204 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 74051 T204 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 74051 T204 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 74052 T204 oasc.RequestHandlers.initHandlersFromConfig created /update: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 74052 T204 oasc.RequestHandlers.initHandlersFromConfig created /update/json: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 74053 T204 oasc.RequestHandlers.initHandlersFromConfig created /update/csv: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 74054 T204 oasc.RequestHandlers.initHandlersFromConfig created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 74055 T204 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 74056 T204 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 74056 T204 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 74056 T204 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 74057 T204 oasc.RequestHandlers.initHandlersFromConfig WARN Multiple requestHandler registered to the same name: /update ignoring: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 74057 T204 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 74058 T204 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 74059 T204 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
   [junit4]   2> 74059 T204 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
   [junit4]   2> 74081 T204 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 74084 T204 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 74086 T204 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 74088 T204 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 74090 T204 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 74095 T204 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 74095 T204 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 74096 T204 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=3, maxMergeAtOnceExplicit=6, maxMergedSegmentMB=56.107421875, floorSegmentMB=0.4912109375, forceMergeDeletesPctAllowed=26.23316267623028, segmentsPerTier=12.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.28139084101318934
   [junit4]   2> 74099 T204 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-A07445A293ED20EE-001/tempDir-001/jetty1/index,segFN=segments_1,generation=1}
   [junit4]   2> 74099 T204 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 74100 T204 oass.SolrIndexSearcher.<init> Opening Searcher@2fe00fed[collection1] main
   [junit4]   2> 74100 T204 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
   [junit4]   2> 74101 T204 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 74102 T204 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 74102 T204 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 74103 T204 oasr.RestManager.init Initializing RestManager with initArgs: {}
   [junit4]   2> 74103 T204 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 74104 T204 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream Read 38 bytes from znode /configs/conf1/_rest_managed.json
   [junit4]   2> 74105 T204 oasr.ManagedResourceStorage.load Loaded LinkedHashMap at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 74105 T204 oasr.ManagedResource.reloadFromStorage Loaded initArgs {} for /rest/managed
   [junit4]   2> 74105 T204 oasr.RestManager.init Initializing 0 registered ManagedResources
   [junit4]   2> 74111 T205 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2fe00fed[collection1] main{UninvertingDirectoryReader()}
   [junit4]   2> 74118 T204 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 74119 T208 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:45461/pb_ collection:collection1 shard:shard2
   [junit4]   2> 74120 T154 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0
   [junit4]   2> 74120 T154 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 74120 T208 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
   [junit4]   2> 74127 T208 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard2
   [junit4]   2> 74130 T180 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 74130 T208 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
   [junit4]   2> 74131 T208 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
   [junit4]   2> ASYNC  NEW_CORE C50 name=collection1 org.apache.solr.core.SolrCore@47c1facb url=http://127.0.0.1:45461/pb_/collection1 node=127.0.0.1:45461_pb_ C50_STATE=coll:collection1 core:collection1 props:{state=down, base_url=http://127.0.0.1:45461/pb_, core=collection1, node_name=127.0.0.1:45461_pb_}
   [junit4]   2> 74131 T208 C50 P45461 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:45461/pb_/collection1/
   [junit4]   2> 74131 T208 C50 P45461 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
   [junit4]   2> 74132 T208 C50 P45461 oasc.SyncStrategy.syncToMe http://127.0.0.1:45461/pb_/collection1/ has no replicas
   [junit4]   2> 74132 T177 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 74132 T208 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:45461/pb_/collection1/ shard2
   [junit4]   2> 74132 T208 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
   [junit4]   2> 74135 T180 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 74136 T180 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 74137 T187 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 74136 T203 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 74142 T177 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 74145 T180 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 74248 T203 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 74248 T187 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 74248 T180 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 74293 T208 oasc.ZkController.register We are http://127.0.0.1:45461/pb_/collection1/ and leader is http://127.0.0.1:45461/pb_/collection1/
   [junit4]   2> 74294 T208 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:45461/pb_
   [junit4]   2> 74294 T208 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
   [junit4]   2> 74294 T208 oasc.ZkController.publish publishing core=collection1 state=active collection=collection1
   [junit4]   2> 74294 T208 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 74297 T180 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 74297 T189 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 74297 T208 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 74297 T188 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 74299 T177 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 74300 T177 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"active",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:45461/pb_",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:45461_pb_",
   [junit4]   2> 	  "shard":"shard2",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "core_node_name":"core_node1"}
   [junit4]   2> 74303 T188 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 74406 T188 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 74406 T203 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 74406 T187 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 75674 T154 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 2
   [junit4]   2> 75675 T154 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 75677 T154 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:41438
   [junit4]   2> 75678 T154 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 75678 T154 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 75678 T154 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-A07445A293ED20EE-001/tempDir-004
   [junit4]   2> 75678 T154 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-A07445A293ED20EE-001/tempDir-004/'
   [junit4]   2> 75711 T154 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-A07445A293ED20EE-001/tempDir-004/solr.xml
   [junit4]   2> 75774 T154 oasc.CoreContainer.<init> New CoreContainer 1040609761
   [junit4]   2> 75774 T154 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-A07445A293ED20EE-001/tempDir-004/]
   [junit4]   2> 75775 T154 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 75776 T154 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 75776 T154 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 75776 T154 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 75776 T154 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 75777 T154 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 75777 T154 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 75777 T154 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 75777 T154 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 75778 T154 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
   [junit4]   2> 75778 T154 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 75778 T154 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 75779 T154 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 75779 T154 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:40090/solr
   [junit4]   2> 75779 T154 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 75780 T154 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 75782 T219 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@191e8a97 name:ZooKeeperConnection Watcher:127.0.0.1:40090 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 75782 T154 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 75785 T154 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 75795 T221 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@562cf72d name:ZooKeeperConnection Watcher:127.0.0.1:40090/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 75795 T154 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 75802 T154 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 76806 T154 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:41438_pb_
   [junit4]   2> 76807 T154 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:41438_pb_
   [junit4]   2> 76819 T223 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
   [junit4]   2> 76819 T223 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 76820 T188 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 76820 T180 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 76820 T189 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 76820 T223 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 76822 T177 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 76823 T177 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:41438/pb_",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:41438_pb_",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "core_node_name":null}
   [junit4]   2> 76823 T177 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
   [junit4]   2> 76823 T177 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
   [junit4]   2> 76826 T189 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 76827 T189 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 76827 T203 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 76827 T187 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 76827 T222 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 77821 T223 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
   [junit4]   2> 77822 T223 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
   [junit4]   2> 77823 T223 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 77823 T223 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
   [junit4]   2> 77824 T223 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 77824 T223 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-A07445A293ED20EE-001/tempDir-004/collection1/'
   [junit4]   2> 77827 T223 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-A07445A293ED20EE-001/tempDir-004/collection1/lib/README' to classloader
   [junit4]   2> 77828 T223 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-A07445A293ED20EE-001/tempDir-004/collection1/lib/classes/' to classloader
   [junit4]   2> 77828 T223 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-A07445A293ED20EE-001/tempDir-004/collection1/lib/.svn/' to classloader
   [junit4]   2> 77902 T223 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_5_0
   [junit4]   2> 77956 T223 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 77958 T223 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 77983 T223 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 78440 T223 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 78444 T223 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 78446 T223 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 78452 T223 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 78496 T223 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection collection1
   [junit4]   2> 78496 T223 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
   [junit4]   2> 78497 T223 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-A07445A293ED20EE-001/tempDir-004/collection1/, dataDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-A07445A293ED20EE-001/tempDir-001/jetty2/
   [junit4]   2> 78497 T223 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@54b83713
   [junit4]   2> 78499 T223 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-A07445A293ED20EE-001/tempDir-001/jetty2
   [junit4]   2> 78500 T223 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-A07445A293ED20EE-001/tempDir-001/jetty2/index/
   [junit4]   2> 78501 T223 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-A07445A293ED20EE-001/tempDir-001/jetty2/index' doesn't exist. Creating new index...
   [junit4]   2> 78502 T223 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-A07445A293ED20EE-001/tempDir-001/jetty2/index
   [junit4]   2> 78503 T223 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=50, maxMergeAtOnceExplicit=16, maxMergedSegmentMB=0.4306640625, floorSegmentMB=1.0390625, forceMergeDeletesPctAllowed=3.007953945489976, segmentsPerTier=41.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 78515 T223 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-A07445A293ED20EE-001/tempDir-001/jetty2/index,segFN=segments_1,generation=1}
   [junit4]   2> 78515 T223 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 78525 T223 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 78525 T223 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 78526 T223 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 78527 T223 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 78527 T223 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 78527 T223 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 78528 T223 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 78528 T223 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 78529 T223 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 78530 T223 oasc.RequestHandlers.initHandlersFromConfig created /update: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 78530 T223 oasc.RequestHandlers.initHandlersFromConfig created /update/json: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 78531 T223 oasc.RequestHandlers.initHandlersFromConfig created /update/csv: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 78532 T223 oasc.RequestHandlers.initHandlersFromConfig created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 78533 T223 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 78533 T223 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 78534 T223 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 78534 T223 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 78535 T223 oasc.RequestHandlers.initHandlersFromConfig WARN Multiple requestHandler registered to the same name: /update ignoring: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 78535 T223 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 78535 T223 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 78536

[...truncated too long message...]

it4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:299)
   [junit4]   2> 	at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:317)
   [junit4]   2> 	at org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:85)
   [junit4]   2> 	at org.apache.solr.cloud.Overseer$ClusterStateUpdater$1.run(Overseer.java:289)
   [junit4]   2> 
   [junit4]   2> 4118055 T16920 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
   [junit4]   2> 4118057 T15349 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/m_gef/be,null}
   [junit4]   2> 4120433 T15349 oas.SolrTestCaseJ4.tearDown ###Ending testDistribSearch
   [junit4]   2> 4120435 T15349 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:39587 39587
   [junit4]   2> 4120617 T15350 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:39587 39587
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=MultiThreadedOCPTest -Dtests.method=testDistribSearch -Dtests.seed=A07445A293ED20EE -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/hudson/lucene-data/enwiki.random.lines.txt -Dtests.locale=zh_TW -Dtests.timezone=Europe/Stockholm -Dtests.file.encoding=US-ASCII
   [junit4] FAILURE 57.0s J0 | MultiThreadedOCPTest.testDistribSearch <<<
   [junit4]    > Throwable #1: java.lang.AssertionError: We have a failed SPLITSHARD task
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([A07445A293ED20EE:2192CBBAE4B240D2]:0)
   [junit4]    > 	at org.apache.solr.cloud.MultiThreadedOCPTest.testTaskExclusivity(MultiThreadedOCPTest.java:125)
   [junit4]    > 	at org.apache.solr.cloud.MultiThreadedOCPTest.doTest(MultiThreadedOCPTest.java:71)
   [junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:865)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 4120628 T15349 oas.SolrTestCaseJ4.deleteCore ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./temp/solr.cloud.MultiThreadedOCPTest-A07445A293ED20EE-001
   [junit4]   2> 56998 T15348 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 1 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=SimpleText, sim=RandomSimilarityProvider(queryNorm=false,coord=no): {}, locale=zh_TW, timezone=Europe/Stockholm
   [junit4]   2> NOTE: FreeBSD 9.1-RELEASE-p3 amd64/Oracle Corporation 1.7.0_60 (64-bit)/cpus=16,threads=1,free=210295832,total=467140608
   [junit4]   2> NOTE: All tests run in this JVM: [DirectSolrSpellCheckerTest, HdfsCollectionsAPIDistributedZkTest, TermsComponentTest, EchoParamsTest, TestDefaultSearchFieldResource, ZkControllerTest, TestAnalyzedSuggestions, TestStressVersions, TestFieldTypeCollectionResource, ChaosMonkeyNothingIsSafeTest, SuggesterWFSTTest, TestCollationFieldDocValues, RAMDirectoryFactoryTest, CoreAdminRequestStatusTest, SolrCoreCheckLockOnStartupTest, OverseerTest, PrimUtilsTest, DeleteInactiveReplicaTest, TestHighlightDedupGrouping, PrimitiveFieldTypeTest, HdfsSyncSliceTest, ChangedSchemaMergeTest, CursorMarkTest, TestManagedSynonymFilterFactory, ClusterStateUpdateTest, XsltUpdateRequestHandlerTest, TestDefaultSimilarityFactory, AnalyticsMergeStrategyTest, TestBinaryResponseWriter, PeerSyncTest, TestComponentsName, UnloadDistributedZkTest, TestFoldingMultitermQuery, PluginInfoTest, TestCodecSupport, TestStressRecovery, TestInfoStreamLogging, PingRequestHandlerTest, XmlUpdateRequestHandlerTest, TestDFRSimilarityFactory, DirectUpdateHandlerTest, SpellCheckCollatorTest, SolrCmdDistributorTest, TestFastOutputStream, DocumentBuilderTest, ReplicationFactorTest, MBeansHandlerTest, TestSolrDeletionPolicy1, TestUtils, TestSolrIndexConfig, ZkNodePropsTest, TestSolrDeletionPolicy2, AssignTest, AbstractAnalyticsStatsTest, TestExceedMaxTermLength, TestSolr4Spatial, TestFastLRUCache, SuggestComponentTest, BadIndexSchemaTest, TestSolrXmlPersistence, SimpleFacetsTest, OverseerCollectionProcessorTest, TestMissingGroups, ReturnFieldsTest, MoreLikeThisHandlerTest, FastVectorHighlighterTest, NoCacheHeaderTest, TestStressReorder, TestPartialUpdateDeduplication, JsonLoaderTest, ShardRoutingTest, TestFastWriter, HdfsBasicDistributedZk2Test, TestSchemaVersionResource, DocExpirationUpdateProcessorFactoryTest, TestFuzzyAnalyzedSuggestions, TestFieldResource, TestSchemaResource, SolrIndexSplitterTest, SuggesterTSTTest, TestAnalyzeInfixSuggestions, TimeZoneUtilsTest, TestLuceneMatchVersion, ExternalFileFieldSortTest, DisMaxRequestHandlerTest, AddSchemaFieldsUpdateProcessorFactoryTest, TestComplexPhraseQParserPlugin, TestManagedResource, HighlighterMaxOffsetTest, TestReplicationHandlerBackup, TestLeaderElectionZkExpiry, TestJmxIntegration, TestCopyFieldCollectionResource, SolrXmlInZkTest, ShowFileRequestHandlerTest, HighlighterConfigTest, HdfsWriteToMultipleCollectionsTest, UpdateParamsTest, RegexBoostProcessorTest, TestSolrXml, TestAddFieldRealTimeGet, OpenExchangeRatesOrgProviderTest, BasicDistributedZkTest, ChaosMonkeySafeLeaderTest, BasicDistributedZk2Test, SyncSliceTest, TestRandomFaceting, LeaderElectionTest, TestRandomDVFaceting, ZkCLITest, ShardRoutingCustomTest, TestDistributedSearch, TestDistributedGrouping, TestFaceting, TestHashPartitioner, DistributedSpellCheckComponentTest, TestRealTimeGet, TestReload, DistributedTermsComponentTest, TestRangeQuery, SpellCheckComponentTest, TestGroupingSearch, QueryElevationComponentTest, TestSort, BasicFunctionalityTest, HighlighterTest, CurrencyFieldOpenExchangeTest, TestCoreDiscovery, AnalysisAfterCoreReloadTest, SignatureUpdateProcessorFactoryTest, TestExtendedDismaxParser, CoreAdminHandlerTest, SuggesterTest, TestTrie, PolyFieldTest, WordBreakSolrSpellCheckerTest, TestPseudoReturnFields, FieldMutatingUpdateProcessorTest, TestWordDelimiterFilterFactory, QueryEqualityTest, DefaultValueUpdateProcessorTest, StatelessScriptUpdateProcessorFactoryTest, TestRemoteStreaming, DocValuesMultiTest, DistanceFunctionTest, SolrInfoMBeanTest, DebugComponentTest, CacheHeaderTest, IndexBasedSpellCheckerTest, TestSurroundQueryParser, LukeRequestHandlerTest, TestQueryUtils, StandardRequestHandlerTest, TestWriterPerf, TestReversedWildcardFilterFactory, DocumentAnalysisRequestHandlerTest, TestQueryTypes, TestOmitPositions, FileBasedSpellCheckerTest, TestValueSourceCache, PathHierarchyTokenizerFactoryTest, RequiredFieldsTest, FieldAnalysisRequestHandlerTest, TestArbitraryIndexDir, IndexSchemaRuntimeFieldTest, LoggingHandlerTest, IndexSchemaTest, UpdateRequestProcessorFactoryTest, JSONWriterTest, TestCSVResponseWriter, UniqFieldsUpdateProcessorFactoryTest, BinaryUpdateRequestHandlerTest, CoreContainerCoreInitFailuresTest, TestLFUCache, SOLR749Test, TestQuerySenderListener, TestQuerySenderNoQuery, CopyFieldTest, ResponseLogComponentTest, SolrIndexConfigTest, BadComponentTest, SampleTest, TestDocSet, TestSearchPerf, NumericFieldsTest, MinimalSchemaTest, TestConfig, TestSolrCoreProperties, DirectSolrConnectionTest, NotRequiredUniqueKeyTest, TestPhraseSuggestions, SpellPossibilityIteratorTest, TestCharFilters, SynonymTokenizerTest, TestXIncludeConfig, TestSweetSpotSimilarityFactory, TestLMDirichletSimilarityFactory, TestLMJelinekMercerSimilarityFactory, TestBM25SimilarityFactory, TestJmxMonitoredMap, ResourceLoaderTest, ScriptEngineTest, URLClassifyProcessorTest, PreAnalyzedFieldTest, DateFieldTest, SpellingQueryConverterTest, DOMUtilTest, ClusterStateTest, TestSolrJ, TestDocumentBuilder, SliceStateTest, SystemInfoHandlerTest, FileUtilsTest, TestRTGBase, CursorPagingTest, TestCursorMarkWithoutUniqueKey, TestDistributedMissingSort, TestTolerantSearch, NoFacetTest, ExpressionTest, AbstractAnalyticsFacetTest, FieldFacetExtrasTest, FieldFacetTest, RangeFacetTest, FunctionTest, CustomCollectionTest, DistribDocExpirationUpdateProcessorTest, MultiThreadedOCPTest]
   [junit4] Completed on J0 in 57.28s, 1 test, 1 failure <<< FAILURES!

[...truncated 234 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:481: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:454: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:45: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/extra-targets.xml:37: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build.xml:189: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/common-build.xml:496: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/common-build.xml:1298: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/common-build.xml:922: There were test failures: 422 suites, 1761 tests, 2 suite-level errors, 1 failure, 51 ignored (9 assumptions)

Total time: 200 minutes 4 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Sending artifact delta relative to Lucene-Solr-NightlyTests-trunk #592
Archived 3 artifacts
Archive block size is 32768
Received 0 blocks and 719889489 bytes
Compression is 0.0%
Took 2 min 5 sec
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-NightlyTests-trunk - Build # 600 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-trunk/600/

1 tests failed.
REGRESSION:  org.apache.lucene.analysis.core.TestRandomChains.testRandomChainsWithLargeStrings

Error Message:
startOffset must be non-negative, and endOffset must be >= startOffset, startOffset=12,endOffset=9

Stack Trace:
java.lang.IllegalArgumentException: startOffset must be non-negative, and endOffset must be >= startOffset, startOffset=12,endOffset=9
	at __randomizedtesting.SeedInfo.seed([A12AB084348D4A48:CB710F956DC36ABB]:0)
	at org.apache.lucene.analysis.tokenattributes.PackedTokenAttributeImpl.setOffset(PackedTokenAttributeImpl.java:107)
	at org.apache.lucene.analysis.shingle.ShingleFilter.incrementToken(ShingleFilter.java:345)
	at org.apache.lucene.analysis.ValidatingTokenFilter.incrementToken(ValidatingTokenFilter.java:68)
	at org.apache.lucene.analysis.BaseTokenStreamTestCase.checkAnalysisConsistency(BaseTokenStreamTestCase.java:704)
	at org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:615)
	at org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:513)
	at org.apache.lucene.analysis.core.TestRandomChains.testRandomChainsWithLargeStrings(TestRandomChains.java:925)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 6509 lines...]
   [junit4] Suite: org.apache.lucene.analysis.core.TestRandomChains
   [junit4]   2> TEST FAIL: useCharFilter=false text='\ud834\uddbd\ud834\uddb6\ud834\udd2f\ud834\udd56 \ue09d\ue77e[\uda4b\udd19\u05c2W\u2444 </script> fybws \u298e\ud9a6\udf66\u2d61\ucbaf\u03da\ue191*\u04b7\u92d3\uf11aH\u0df6\u03f9 &a'
   [junit4]   2> Exception from random analyzer: 
   [junit4]   2> charfilters=
   [junit4]   2>   org.apache.lucene.analysis.charfilter.MappingCharFilter(org.apache.lucene.analysis.charfilter.NormalizeCharMap@1d2a5506, java.io.StringReader@46340bdf)
   [junit4]   2>   org.apache.lucene.analysis.fa.PersianCharFilter(org.apache.lucene.analysis.charfilter.MappingCharFilter@4755b3c2)
   [junit4]   2> tokenizer=
   [junit4]   2>   org.apache.lucene.analysis.path.PathHierarchyTokenizer()
   [junit4]   2> filters=
   [junit4]   2>   org.apache.lucene.analysis.miscellaneous.ASCIIFoldingFilter(ValidatingTokenFilter@7ff6483b term=,bytes=[],startOffset=0,endOffset=0,positionIncrement=1,positionLength=1,type=word)
   [junit4]   2>   org.apache.lucene.analysis.ar.ArabicNormalizationFilter(ValidatingTokenFilter@655e513f term=,bytes=[],startOffset=0,endOffset=0,positionIncrement=1,positionLength=1,type=word)
   [junit4]   2>   org.apache.lucene.analysis.miscellaneous.WordDelimiterFilter(ValidatingTokenFilter@343234d4 term=,bytes=[],startOffset=0,endOffset=0,positionIncrement=1,positionLength=1,type=word, [-123, -53, -19, 86, 30, -43, -10, -41, -17, -104, -21, 47, 39, -69, -9, 90, 36, 114, 75, -69, 77, -121, 40, 49, -37, -23, -30, 51, -106, -66, -61, 104, -127, -38, 111, -88, -43, -30, -128, -71, -107, -2, -9, -11, -115, -93, 2, 102, -119, 54, -82, 73, -58], 49, [igagfch, fzece, wdmbpkqopi, mchxdcl, snt, yipj, knrqo, n])
   [junit4]   2>   org.apache.lucene.analysis.shingle.ShingleFilter(ValidatingTokenFilter@73516627 term=,bytes=[],startOffset=0,endOffset=0,positionIncrement=1,positionLength=1,type=word, 41)
   [junit4]   2> offsetsAreCorrect=false
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestRandomChains -Dtests.method=testRandomChainsWithLargeStrings -Dtests.seed=A12AB084348D4A48 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/hudson/lucene-data/enwiki.random.lines.txt -Dtests.locale=zh_CN -Dtests.timezone=Australia/Melbourne -Dtests.file.encoding=US-ASCII
   [junit4] ERROR   8.01s J1 | TestRandomChains.testRandomChainsWithLargeStrings <<<
   [junit4]    > Throwable #1: java.lang.IllegalArgumentException: startOffset must be non-negative, and endOffset must be >= startOffset, startOffset=12,endOffset=9
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([A12AB084348D4A48:CB710F956DC36ABB]:0)
   [junit4]    > 	at org.apache.lucene.analysis.tokenattributes.PackedTokenAttributeImpl.setOffset(PackedTokenAttributeImpl.java:107)
   [junit4]    > 	at org.apache.lucene.analysis.shingle.ShingleFilter.incrementToken(ShingleFilter.java:345)
   [junit4]    > 	at org.apache.lucene.analysis.ValidatingTokenFilter.incrementToken(ValidatingTokenFilter.java:68)
   [junit4]    > 	at org.apache.lucene.analysis.BaseTokenStreamTestCase.checkAnalysisConsistency(BaseTokenStreamTestCase.java:704)
   [junit4]    > 	at org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:615)
   [junit4]    > 	at org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:513)
   [junit4]    > 	at org.apache.lucene.analysis.core.TestRandomChains.testRandomChainsWithLargeStrings(TestRandomChains.java:925)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> NOTE: leaving temporary files on disk at: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/build/analysis/common/test/J1/./temp/lucene.analysis.core.TestRandomChains-A12AB084348D4A48-001
   [junit4]   2> NOTE: test params are: codec=Lucene45, sim=RandomSimilarityProvider(queryNorm=true,coord=crazy): {dummy=DFR I(n)B3(800.0)}, locale=zh_CN, timezone=Australia/Melbourne
   [junit4]   2> NOTE: FreeBSD 9.1-RELEASE-p3 amd64/Oracle Corporation 1.7.0_60 (64-bit)/cpus=16,threads=1,free=78087424,total=288358400
   [junit4]   2> NOTE: All tests run in this JVM: [TestRollingCharBuffer, TestHindiAnalyzer, TestTurkishLowerCaseFilterFactory, TestCJKBigramFilter, EdgeNGramTokenizerTest, TestPersianAnalyzer, TestRussianAnalyzer, TestCzechStemmer, TestGreekStemmer, TestGalicianAnalyzer, TestTypeTokenFilter, TestKeepFilterFactory, TestScandinavianFoldingFilterFactory, TestCharTokenizers, TestArmenianAnalyzer, TestGalicianStemFilterFactory, TestSwedishLightStemFilter, DateRecognizerSinkTokenizerTest, TestBrazilianStemmer, TestLimitTokenPositionFilterFactory, TestEnglishAnalyzer, TestCharArrayIterator, TestSpanishLightStemFilterFactory, TestCharArrayMap, CommonGramsFilterTest, TestSnowball, TestDoubleEscape, TestBulgarianStemFilterFactory, TestHunspellStemFilterFactory, TestCJKWidthFilter, TestNorwegianMinimalStemFilter, TestLatvianStemFilterFactory, TestStemmer, TestDictionaryCompoundWordTokenFilterFactory, TestCodepointCountFilter, TestPortugueseAnalyzer, TokenOffsetPayloadTokenFilterTest, TestPortugueseMinimalStemFilterFactory, TestSpaces, TestEscaped, TestSpanishAnalyzer, ShingleFilterTest, TestGreekStemFilterFactory, QueryAutoStopWordAnalyzerTest, TestCompoundWordTokenFilter, TestZeroAffix2, TestHungarianLightStemFilterFactory, TestStandardAnalyzer, TestKeywordMarkerFilter, TestKStemmer, TestUAX29URLEmailTokenizer, TestTurkishAnalyzer, TestCommonGramsQueryFilterFactory, TestKeywordRepeatFilter, TestCapitalizationFilter, TestHindiFilters, TestAnalysisSPILoader, TestFrenchLightStemFilter, TestStemmerOverrideFilterFactory, TestPersianCharFilter, TestWordnetSynonymParser, TestSpanishLightStemFilter, TestFinnishLightStemFilterFactory, TestGermanStemFilterFactory, TestCollationDocValuesField, TestTruncateTokenFilter, TestFactories, TestRandomChains]
   [junit4] Completed on J1 in 55.45s, 2 tests, 1 error <<< FAILURES!

[...truncated 407 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:481: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:454: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:45: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/extra-targets.xml:37: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/build.xml:544: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/common-build.xml:2028: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/analysis/build.xml:106: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/analysis/build.xml:38: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/module-build.xml:60: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/common-build.xml:1298: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/common-build.xml:922: There were test failures: 268 suites, 1358 tests, 1 error, 1 ignored

Total time: 81 minutes 28 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Sending artifact delta relative to Lucene-Solr-NightlyTests-trunk #592
Archived 3 artifacts
Archive block size is 32768
Received 0 blocks and 1669841 bytes
Compression is 0.0%
Took 0.6 sec
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure