You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Charlie Cron <hu...@gmail.com> on 2012/10/26 15:02:43 UTC

Build failed in Jenkins: slow-io-beasting #4869

See <http://sierranevada.servebeer.com:8080/job/slow-io-beasting/4869/>

------------------------------------------
[...truncated 18864 lines...]
[junit4:junit4]   2> 13178 T583 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 13178 T583 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 13178 T583 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 13178 T583 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 13178 T583 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 13178 T583 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 13178 T583 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 13178 T583 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 13178 T583 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 13178 T583 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 13178 T583 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 13178 T583 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 13178 T583 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 13225 T583 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 13256 T583 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 13256 T583 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 13271 T583 oass.SolrIndexSearcher.<init> Opening Searcher@1914fe9 main
[junit4:junit4]   2> 13271 T583 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 13271 T583 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 13271 T583 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 13271 T583 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 13271 T583 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 13287 T609 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1914fe9 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 13287 T583 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 13583 T595 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 13583 T595 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":null,
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"BIGBOY:8983_solr",
[junit4:junit4]   2> 	  "base_url":"http://BIGBOY:8983/solr"}
[junit4:junit4]   2> 13583 T594 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (3)
[junit4:junit4]   2> 14753 T586 oazs.ZooKeeperServer.expire Expiring session 0x13a9d2773470003, timeout of 3000ms exceeded
[junit4:junit4]   2> 14753 T588 oazs.PrepRequestProcessor.pRequest Processed session termination for sessionid: 0x13a9d2773470003
[junit4:junit4]   2> 14769 T594 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 14769 T608 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 75019 T583 oasc.SolrException.log SEVERE null:org.apache.solr.common.SolrException: Could not get shard_id for core: collection1
[junit4:junit4]   2> 		at org.apache.solr.cloud.ZkController.doGetShardIdProcess(ZkController.java:996)
[junit4:junit4]   2> 		at org.apache.solr.cloud.ZkController.preRegister(ZkController.java:1054)
[junit4:junit4]   2> 		at org.apache.solr.core.CoreContainer.register(CoreContainer.java:657)
[junit4:junit4]   2> 		at org.apache.solr.core.CoreContainer.load(CoreContainer.java:530)
[junit4:junit4]   2> 		at org.apache.solr.core.CoreContainer$Initializer.initialize(CoreContainer.java:306)
[junit4:junit4]   2> 		at org.apache.solr.cloud.ClusterStateUpdateTest.testCoreRegistration(ClusterStateUpdateTest.java:223)
[junit4:junit4]   2> 		at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4:junit4]   2> 		at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
[junit4:junit4]   2> 		at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
[junit4:junit4]   2> 		at java.lang.reflect.Method.invoke(Method.java:597)
[junit4:junit4]   2> 		at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
[junit4:junit4]   2> 		at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
[junit4:junit4]   2> 		at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
[junit4:junit4]   2> 		at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
[junit4:junit4]   2> 		at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
[junit4:junit4]   2> 		at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
[junit4:junit4]   2> 		at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
[junit4:junit4]   2> 		at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
[junit4:junit4]   2> 		at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4:junit4]   2> 		at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
[junit4:junit4]   2> 		at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
[junit4:junit4]   2> 		at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
[junit4:junit4]   2> 		at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4:junit4]   2> 		at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4]   2> 		at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
[junit4:junit4]   2> 		at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
[junit4:junit4]   2> 		at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
[junit4:junit4]   2> 		at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
[junit4:junit4]   2> 		at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
[junit4:junit4]   2> 		at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
[junit4:junit4]   2> 		at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
[junit4:junit4]   2> 		at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4]   2> 		at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
[junit4:junit4]   2> 		at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4:junit4]   2> 		at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
[junit4:junit4]   2> 		at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
[junit4:junit4]   2> 		at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4:junit4]   2> 		at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4:junit4]   2> 		at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4]   2> 		at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
[junit4:junit4]   2> 		at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4:junit4]   2> 		at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
[junit4:junit4]   2> 		at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
[junit4:junit4]   2> 		at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4]   2> 		at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
[junit4:junit4]   2> 		at java.lang.Thread.run(Thread.java:662)
[junit4:junit4]   2> 	
[junit4:junit4]   2> 75019 T583 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=15397978
[junit4:junit4]   2> 75019 T583 oasc.SolrCore.close [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@12e1b7f
[junit4:junit4]   2> 75019 T583 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
[junit4:junit4]   2> 75019 T583 oasc.SolrCore.decrefSolrCoreState Closing SolrCoreState
[junit4:junit4]   2> 75019 T583 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4]   2> 75019 T583 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4:junit4]   2> 75019 T583 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4:junit4]   2> 75019 T583 oasc.SolrCore.close [testcore]  CLOSING SolrCore org.apache.solr.core.SolrCore@2925e
[junit4:junit4]   2> 75019 T583 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
[junit4:junit4]   2> 75019 T583 oasc.SolrCore.decrefSolrCoreState Closing SolrCoreState
[junit4:junit4]   2> 75019 T583 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4]   2> 75019 T583 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4:junit4]   2> 75019 T583 oasc.SolrCore.closeSearcher [testcore] Closing main searcher on request.
[junit4:junit4]   2> 75019 T595 oasc.Overseer$ClusterStateUpdater.amILeader According to ZK I (id=88555935450464258-127.0.0.1:1661_solr-n_0000000000) am no longer a leader.
[junit4:junit4]   2> 75019 T588 oazs.PrepRequestProcessor.pRequest Processed session termination for sessionid: 0x13a9d2773470002
[junit4:junit4]   2> 75034 T583 oaz.ZooKeeper.close Session: 0x13a9d2773470002 closed
[junit4:junit4]   2> 75034 T585 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:52037 which had sessionid 0x13a9d2773470002
[junit4:junit4]   2> 75034 T583 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=4248874
[junit4:junit4]   2> 75034 T588 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13a9d2773470006 type:delete cxid:0x39 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer_elect/leader Error:KeeperErrorCode = NoNode for /solr/overseer_elect/leader
[junit4:junit4]   2> 75034 T588 oazs.PrepRequestProcessor.pRequest Processed session termination for sessionid: 0x13a9d2773470006
[junit4:junit4]   2> 75034 T608 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4]   2> 75034 T583 oaz.ZooKeeper.close Session: 0x13a9d2773470006 closed
[junit4:junit4]   2> 75034 T585 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:52058 which had sessionid 0x13a9d2773470006
[junit4:junit4]   2> 75034 T583 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=11695326
[junit4:junit4]   2> 75034 T588 oazs.PrepRequestProcessor.run PrepRequestProcessor exited loop!
[junit4:junit4]   2> 75034 T587 oazs.SyncRequestProcessor.run SyncRequestProcessor exited!
[junit4:junit4]   2> 75034 T583 oazs.FinalRequestProcessor.shutdown shutdown of request processor complete
[junit4:junit4]   2> 75752 T586 oazs.SessionTrackerImpl.run SessionTrackerImpl exited loop!
[junit4:junit4]   2> 76048 T594 oascc.ZkStateReader$3.process WARNING ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4]   2> 76048 T608 oasc.LeaderElector$1.process WARNING  org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect
[junit4:junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:118)
[junit4:junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:42)
[junit4:junit4]   2> 	at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:815)
[junit4:junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$3.execute(SolrZkClient.java:176)
[junit4:junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$3.execute(SolrZkClient.java:173)
[junit4:junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:63)
[junit4:junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:173)
[junit4:junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:384)
[junit4:junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:353)
[junit4:junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:340)
[junit4:junit4]   2> 	at org.apache.solr.cloud.OverseerElectionContext.runLeaderProcess(ElectionContext.java:383)
[junit4:junit4]   2> 	at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:155)
[junit4:junit4]   2> 	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:99)
[junit4:junit4]   2> 	at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
[junit4:junit4]   2> 	at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:128)
[junit4:junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:526)
[junit4:junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502)
[junit4:junit4]   2> 
[junit4:junit4]   2> 77327 T594 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 77327 T585 oazs.NIOServerCnxn$Factory.run WARNING Ignoring unexpected runtime exception java.nio.channels.CancelledKeyException
[junit4:junit4]   2> 	at sun.nio.ch.SelectionKeyImpl.ensureValid(SelectionKeyImpl.java:55)
[junit4:junit4]   2> 	at sun.nio.ch.SelectionKeyImpl.readyOps(SelectionKeyImpl.java:69)
[junit4:junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn$Factory.run(NIOServerCnxn.java:241)
[junit4:junit4]   2> 
[junit4:junit4]   2> 77327 T608 oascc.ZkStateReader$3.process WARNING ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4]   2> 77327 T585 oazs.NIOServerCnxn$Factory.run NIOServerCnxn factory exited run method
[junit4:junit4]   2> 77327 T608 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 77327 T583 oazs.FinalRequestProcessor.shutdown shutdown of request processor complete
[junit4:junit4]   2> 77327 T583 oas.SolrTestCaseJ4.tearDown ###Ending testCoreRegistration
[junit4:junit4]   2> 77327 T583 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4:junit4]   2> 200146 T583 oas.SolrTestCaseJ4.endTrackingSearchers SEVERE ERROR: SolrIndexSearcher opens=5 closes=4
[junit4:junit4]   2> 200146 T582 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 1 leaked thread(s).
[junit4:junit4]   2> 220301 T582 ccr.ThreadLeakControl.checkThreadLeaks SEVERE 1 thread leaked from SUITE scope at org.apache.solr.cloud.ClusterStateUpdateTest: 
[junit4:junit4]   2> 	   1) Thread[id=609, name=searcherExecutor-387-thread-1, state=WAITING, group=TGRP-ClusterStateUpdateTest]
[junit4:junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
[junit4:junit4]   2> 	        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:662)
[junit4:junit4]   2> 220301 T582 ccr.ThreadLeakControl.tryToInterruptAll Starting to interrupt leaked threads:
[junit4:junit4]   2> 	   1) Thread[id=609, name=searcherExecutor-387-thread-1, state=WAITING, group=TGRP-ClusterStateUpdateTest]
[junit4:junit4]   2> 223390 T582 ccr.ThreadLeakControl.tryToInterruptAll SEVERE There are still zombie threads that couldn't be terminated:
[junit4:junit4]   2> 	   1) Thread[id=609, name=searcherExecutor-387-thread-1, state=WAITING, group=TGRP-ClusterStateUpdateTest]
[junit4:junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
[junit4:junit4]   2> 	        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:662)
[junit4:junit4]   2> NOTE: test params are: codec=Lucene3x, sim=DefaultSimilarity, locale=ar, timezone=America/Bahia
[junit4:junit4]   2> NOTE: Windows Vista 6.0 x86/Sun Microsystems Inc. 1.6.0_23 (32-bit)/cpus=4,threads=2,free=18566344,total=53583872
[junit4:junit4]   2> NOTE: All tests run in this JVM: [TestPHPSerializedResponseWriter, LeaderElectionTest, ResourceLoaderTest, TestAnalyzedSuggestions, BinaryUpdateRequestHandlerTest, CacheHeaderTest, BadIndexSchemaTest, PeerSyncTest, DistributedSpellCheckComponentTest, SolrIndexConfigTest, TestSurroundQueryParser, CopyFieldTest, UUIDFieldTest, TestElisionMultitermQuery, TestFunctionQuery, TestCoreContainer, TermVectorComponentDistributedTest, StatsComponentTest, SolrCoreTest, ConvertedLegacyTest, SampleTest, SolrRequestParserTest, MoreLikeThisHandlerTest, IndexSchemaTest, XmlUpdateRequestHandlerTest, TestSolrQueryParser, FastVectorHighlighterTest, TestArbitraryIndexDir, SuggesterTSTTest, TestSearchPerf, NotRequiredUniqueKeyTest, TestLuceneMatchVersion, NoCacheHeaderTest, TestPropInjectDefaults, IndexReaderFactoryTest, EchoParamsTest, TestPluginEnable, TestFastLRUCache, PrimUtilsTest, TestSuggestSpellingConverter, ClusterStateUpdateTest]
[junit4:junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=ClusterStateUpdateTest -Dtests.seed=872DF29893E7BABE -Dtests.slow=true -Dtests.locale=ar -Dtests.timezone=America/Bahia -Dtests.file.encoding=Cp1252
[junit4:junit4] ERROR   0.00s J7 | ClusterStateUpdateTest (suite) <<<
[junit4:junit4]    > Throwable #1: java.lang.AssertionError: ERROR: SolrIndexSearcher opens=5 closes=4
[junit4:junit4]    > 	at __randomizedtesting.SeedInfo.seed([872DF29893E7BABE]:0)
[junit4:junit4]    > 	at org.junit.Assert.fail(Assert.java:93)
[junit4:junit4]    > 	at org.apache.solr.SolrTestCaseJ4.endTrackingSearchers(SolrTestCaseJ4.java:242)
[junit4:junit4]    > 	at org.apache.solr.SolrTestCaseJ4.afterClass(SolrTestCaseJ4.java:97)
[junit4:junit4]    > 	at sun.reflect.GeneratedMethodAccessor21.invoke(Unknown Source)
[junit4:junit4]    > 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
[junit4:junit4]    > 	at java.lang.reflect.Method.invoke(Method.java:597)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:700)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
[junit4:junit4]    > 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
[junit4:junit4]    > 	at java.lang.Thread.run(Thread.java:662)
[junit4:junit4]    > Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.cloud.ClusterStateUpdateTest: 
[junit4:junit4]    >    1) Thread[id=609, name=searcherExecutor-387-thread-1, state=WAITING, group=TGRP-ClusterStateUpdateTest]
[junit4:junit4]    >         at sun.misc.Unsafe.park(Native Method)
[junit4:junit4]    >         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
[junit4:junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
[junit4:junit4]    >         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
[junit4:junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
[junit4:junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:662)
[junit4:junit4]    > 	at __randomizedtesting.SeedInfo.seed([872DF29893E7BABE]:0)
[junit4:junit4]    > Throwable #3: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
[junit4:junit4]    >    1) Thread[id=609, name=searcherExecutor-387-thread-1, state=WAITING, group=TGRP-ClusterStateUpdateTest]
[junit4:junit4]    >         at sun.misc.Unsafe.park(Native Method)
[junit4:junit4]    >         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
[junit4:junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
[junit4:junit4]    >         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
[junit4:junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
[junit4:junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:662)
[junit4:junit4]    > 	at __randomizedtesting.SeedInfo.seed([872DF29893E7BABE]:0)
[junit4:junit4] Completed on J7 in 223.45s, 1 test, 1 failure, 2 errors <<< FAILURES!
[junit4:junit4] 
[junit4:junit4] 
[junit4:junit4] Tests with failures:
[junit4:junit4]   - org.apache.solr.handler.TestReplicationHandler (suite)
[junit4:junit4]   - org.apache.solr.cloud.ClusterStateUpdateTest (suite)
[junit4:junit4] 
[junit4:junit4] 
[junit4:junit4] JVM J0:     2.62 ..   266.83 =   264.20s
[junit4:junit4] JVM J1:     2.62 ..   274.56 =   271.94s
[junit4:junit4] JVM J2:     2.37 ..   263.94 =   261.57s
[junit4:junit4] JVM J3:     2.62 ..   257.83 =   255.21s
[junit4:junit4] JVM J4:     2.62 ..   298.71 =   296.09s
[junit4:junit4] JVM J5:     2.62 ..   358.38 =   355.76s
[junit4:junit4] JVM J6:     2.62 ..   259.92 =   257.30s
[junit4:junit4] JVM J7:     2.62 ..   477.68 =   475.06s
[junit4:junit4] Execution time total: 7 minutes 57 seconds
[junit4:junit4] Tests summary: 233 suites, 965 tests, 4 suite-level errors, 12 ignored (7 assumptions)

BUILD FAILED
<http://sierranevada.servebeer.com:8080/job/slow-io-beasting/ws/build.xml>:39: The following error occurred while executing this line:
<http://sierranevada.servebeer.com:8080/job/slow-io-beasting/ws/solr/build.xml>:178: The following error occurred while executing this line:
<http://sierranevada.servebeer.com:8080/job/slow-io-beasting/ws/lucene/module-build.xml>:61: The following error occurred while executing this line:
<http://sierranevada.servebeer.com:8080/job/slow-io-beasting/ws/lucene/common-build.xml>:1171: The following error occurred while executing this line:
<http://sierranevada.servebeer.com:8080/job/slow-io-beasting/ws/lucene/common-build.xml>:839: There were test failures: 233 suites, 965 tests, 4 suite-level errors, 12 ignored (7 assumptions)

Total time: 15 minutes 59 seconds
Build step 'Invoke Ant' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Jenkins build is back to normal : slow-io-beasting #4872

Posted by Charlie Cron <hu...@gmail.com>.
See <http://sierranevada.servebeer.com:8080/job/slow-io-beasting/4872/>


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Build failed in Jenkins: slow-io-beasting #4871

Posted by Charlie Cron <hu...@gmail.com>.
See <http://sierranevada.servebeer.com:8080/job/slow-io-beasting/4871/>

------------------------------------------
[...truncated 17899 lines...]
[junit4:junit4]   2> 77866 T379 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[359]} 0 0
[junit4:junit4]   2> 77882 T377 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[360]} 0 0
[junit4:junit4]   2> 77898 T378 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[361]} 0 0
[junit4:junit4]   2> 77913 T370 oash.SnapPuller.fetchLatestIndex SEVERE Master at: http://127.0.0.1:56093/solr is not available. Index fetch failed. Exception: org.apache.solr.client.solrj.SolrServerException: Server refused connection at: http://127.0.0.1:56093/solr
[junit4:junit4]   2> 77913 T373 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[362]} 0 0
[junit4:junit4]   2> 77929 T375 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[363]} 0 0
[junit4:junit4]   2> 77944 T376 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[364]} 0 0
[junit4:junit4]   2> 77960 T379 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[365]} 0 0
[junit4:junit4]   2> 77976 T377 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[366]} 0 0
[junit4:junit4]   2> 77991 T378 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[367]} 0 0
[junit4:junit4]   2> 78007 T373 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[368]} 0 0
[junit4:junit4]   2> 78022 T375 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[369]} 0 0
[junit4:junit4]   2> 78038 T379 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[370]} 0 0
[junit4:junit4]   2> 78054 T377 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[371]} 0 0
[junit4:junit4]   2> 78069 T378 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[372]} 0 0
[junit4:junit4]   2> 78085 T373 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[373]} 0 0
[junit4:junit4]   2> 78100 T375 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[374]} 0 0
[junit4:junit4]   2> 78116 T376 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[375]} 0 0
[junit4:junit4]   2> 78132 T379 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[376]} 0 0
[junit4:junit4]   2> 78147 T377 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[377]} 0 0
[junit4:junit4]   2> 78163 T378 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[378]} 0 0
[junit4:junit4]   2> 78178 T373 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[379]} 0 0
[junit4:junit4]   2> 78194 T375 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[380]} 0 0
[junit4:junit4]   2> 78210 T376 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[381]} 0 0
[junit4:junit4]   2> 78225 T379 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[382]} 0 0
[junit4:junit4]   2> 78241 T377 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[383]} 0 0
[junit4:junit4]   2> 78256 T378 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[384]} 0 0
[junit4:junit4]   2> 78272 T373 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[385]} 0 0
[junit4:junit4]   2> 78288 T375 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[386]} 0 0
[junit4:junit4]   2> 78303 T376 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[387]} 0 0
[junit4:junit4]   2> 78319 T379 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[388]} 0 0
[junit4:junit4]   2> 78334 T377 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[389]} 0 0
[junit4:junit4]   2> 78350 T378 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[390]} 0 0
[junit4:junit4]   2> 78366 T373 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[391]} 0 0
[junit4:junit4]   2> 78381 T375 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[392]} 0 0
[junit4:junit4]   2> 78397 T376 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[393]} 0 0
[junit4:junit4]   2> 78412 T379 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[394]} 0 0
[junit4:junit4]   2> 78428 T377 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[395]} 0 0
[junit4:junit4]   2> 78444 T373 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[396]} 0 0
[junit4:junit4]   2> 78459 T375 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[397]} 0 0
[junit4:junit4]   2> 78475 T376 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[398]} 0 0
[junit4:junit4]   2> 78490 T379 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[399]} 0 0
[junit4:junit4]   2> 78506 T378 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[400]} 0 0
[junit4:junit4]   2> 78522 T377 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[401]} 0 0
[junit4:junit4]   2> 78537 T373 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[402]} 0 0
[junit4:junit4]   2> 78553 T375 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[403]} 0 0
[junit4:junit4]   2> 78568 T376 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[404]} 0 0
[junit4:junit4]   2> 78584 T379 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[405]} 0 0
[junit4:junit4]   2> 78600 T378 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[406]} 0 0
[junit4:junit4]   2> 78615 T377 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[407]} 0 0
[junit4:junit4]   2> 78631 T373 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[408]} 0 0
[junit4:junit4]   2> 78646 T375 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[409]} 0 0
[junit4:junit4]   2> 78662 T376 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[410]} 0 0
[junit4:junit4]   2> 78678 T379 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[411]} 0 0
[junit4:junit4]   2> 78693 T378 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[412]} 0 0
[junit4:junit4]   2> 78709 T377 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[413]} 0 0
[junit4:junit4]   2> 78724 T373 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[414]} 0 0
[junit4:junit4]   2> 78740 T375 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[415]} 0 0
[junit4:junit4]   2> 78756 T376 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[416]} 0 0
[junit4:junit4]   2> 78771 T379 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[417]} 0 0
[junit4:junit4]   2> 78787 T378 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[418]} 0 0
[junit4:junit4]   2> 78802 T377 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[419]} 0 0
[junit4:junit4]   2> 78818 T373 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[420]} 0 0
[junit4:junit4]   2> 78834 T375 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[421]} 0 0
[junit4:junit4]   2> 78849 T379 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[422]} 0 0
[junit4:junit4]   2> 78865 T378 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[423]} 0 0
[junit4:junit4]   2> 78880 T377 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[424]} 0 0
[junit4:junit4]   2> 78896 T373 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[425]} 0 0
[junit4:junit4]   2> 78912 T375 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[426]} 0 0
[junit4:junit4]   2> 78927 T376 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[427]} 0 0
[junit4:junit4]   2> 78943 T379 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[428]} 0 0
[junit4:junit4]   2> 78944 T378 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[429]} 0 0
[junit4:junit4]   2> 78975 T377 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[430]} 0 0
[junit4:junit4]   2> 78976 T373 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[431]} 0 0
[junit4:junit4]   2> 78977 T370 oash.SnapPuller.fetchLatestIndex SEVERE Master at: http://127.0.0.1:56093/solr is not available. Index fetch failed. Exception: org.apache.solr.client.solrj.SolrServerException: Server refused connection at: http://127.0.0.1:56093/solr
[junit4:junit4]   2> 78978 T375 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[432]} 0 0
[junit4:junit4]   2> 78984 T376 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[433]} 0 0
[junit4:junit4]   2> 78994 T379 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[434]} 0 1
[junit4:junit4]   2> 79029 T378 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[435]} 0 0
[junit4:junit4]   2> 79045 T377 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[436]} 0 0
[junit4:junit4]   2> 79060 T373 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[437]} 0 0
[junit4:junit4]   2> 79076 T375 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[438]} 0 0
[junit4:junit4]   2> 79092 T376 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[439]} 0 0
[junit4:junit4]   2> 79093 T379 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[440]} 0 0
[junit4:junit4]   2> 79101 T378 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[441]} 0 0
[junit4:junit4]   2> 79105 T377 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[442]} 0 0
[junit4:junit4]   2> 79114 T373 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[443]} 0 0
[junit4:junit4]   2> 79119 T375 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[444]} 0 0
[junit4:junit4]   2> 79126 T376 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[445]} 0 0
[junit4:junit4]   2> 79129 T379 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[446]} 0 0
[junit4:junit4]   2> 79133 T378 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[447]} 0 0
[junit4:junit4]   2> 79164 T377 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[448]} 0 0
[junit4:junit4]   2> 79179 T373 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[449]} 0 0
[junit4:junit4]   2> 79195 T375 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[450]} 0 0
[junit4:junit4]   2> 79196 T376 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[451]} 0 0
[junit4:junit4]   2> 79201 T379 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[452]} 0 0
[junit4:junit4]   2> 79205 T378 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[453]} 0 0
[junit4:junit4]   2> 79221 T377 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[454]} 0 0
[junit4:junit4]   2> 79238 T373 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[455]} 0 0
[junit4:junit4]   2> 79270 T375 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[456]} 0 0
[junit4:junit4]   2> 79271 T379 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[457]} 0 0
[junit4:junit4]   2> 79302 T378 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[458]} 0 15
[junit4:junit4]   2> 79318 T377 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[459]} 0 0
[junit4:junit4]   2> 79319 T373 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[460]} 0 0
[junit4:junit4]   2> 79324 T376 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[461]} 0 0
[junit4:junit4]   2> 79329 T375 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[462]} 0 0
[junit4:junit4]   2> 79330 T379 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[463]} 0 0
[junit4:junit4]   2> 79334 T378 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[464]} 0 0
[junit4:junit4]   2> 79336 T377 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[465]} 0 0
[junit4:junit4]   2> 79338 T373 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[466]} 0 0
[junit4:junit4]   2> 79339 T376 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[467]} 0 0
[junit4:junit4]   2> 79342 T375 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[468]} 0 1
[junit4:junit4]   2> 79344 T379 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[469]} 0 1
[junit4:junit4]   2> 79349 T378 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[470]} 0 0
[junit4:junit4]   2> 79393 T377 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[471]} 0 0
[junit4:junit4]   2> 79394 T373 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[472]} 0 0
[junit4:junit4]   2> 79402 T376 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[473]} 0 0
[junit4:junit4]   2> 79419 T375 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[474]} 0 0
[junit4:junit4]   2> 79450 T379 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[475]} 0 0
[junit4:junit4]   2> 79466 T378 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[476]} 0 0
[junit4:junit4]   2> 79481 T377 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[477]} 0 0
[junit4:junit4]   2> 79497 T373 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[478]} 0 0
[junit4:junit4]   2> 79512 T376 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[479]} 0 0
[junit4:junit4]   2> 79528 T375 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[480]} 0 0
[junit4:junit4]   2> 79544 T379 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[481]} 0 0
[junit4:junit4]   2> 79559 T378 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[482]} 0 0
[junit4:junit4]   2> 79575 T377 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[483]} 0 0
[junit4:junit4]   2> 79590 T373 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[484]} 0 0
[junit4:junit4]   2> 79606 T376 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[485]} 0 0
[junit4:junit4]   2> 79622 T375 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[486]} 0 0
[junit4:junit4]   2> 79637 T379 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[487]} 0 0
[junit4:junit4]   2> 79653 T378 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[488]} 0 0
[junit4:junit4]   2> 79668 T373 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[489]} 0 0
[junit4:junit4]   2> 79684 T376 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[490]} 0 0
[junit4:junit4]   2> 79700 T375 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[491]} 0 0
[junit4:junit4]   2> 79731 T379 C49 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[492]} 0 0
[junit4:junit4]   2> 79746 T377 C49 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 79824 T377 C49 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.SimpleFSDirectory@<http://sierranevada.servebeer.com:8080/job/slow-io-beasting/ws/solr/build/solr-core/test/J5/org.apache.solr.handler.TestReplicationHandler$SolrInstance-1351258785491/master/collection1/data/index> lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1b6dce3; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_i,generation=18,filenames=[_a.frq, _a.tii, _a.fdt, _a.si, _a.nrm, segments_i, _a.fnm, _a.tis, _a.fdx]
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.SimpleFSDirectory@<http://sierranevada.servebeer.com:8080/job/slow-io-beasting/ws/solr/build/solr-core/test/J5/org.apache.solr.handler.TestReplicationHandler$SolrInstance-1351258785491/master/collection1/data/index> lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1b6dce3; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_j,generation=19,filenames=[_b.tii, _b.frq, _b.si, _b.fnm, _b.fdt, _b.nrm, _b.tis, segments_j, _b.fdx]
[junit4:junit4]   2> 79824 T377 C49 oasc.SolrDeletionPolicy.updateCommits newest commit = 19
[junit4:junit4]   2> 79856 T377 C49 oass.SolrIndexSearcher.<init> Opening Searcher@a0dde2 main
[junit4:junit4]   2> 79856 T377 C49 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 79856 T381 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@a0dde2 main{StandardDirectoryReader(segments_j:35 _b(4.1):C493)}
[junit4:junit4]   2> 79856 T377 C49 UPDATE [collection1] webapp=/solr path=/update params={waitSearcher=true&wt=javabin&commit=true&softCommit=false&version=2} {commit=} 0 110
[junit4:junit4]   2> 79871 T378 C49 REQ [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
[junit4:junit4]   2> 79902 T373 C49 REQ [collection1] webapp=/solr path=/replication params={command=backup&numberToKeep=1} status=0 QTime=31 
[junit4:junit4]   2> 80074 T376 C49 REQ [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
[junit4:junit4]   2> 80090 T370 oash.SnapPuller.fetchLatestIndex SEVERE Master at: http://127.0.0.1:56093/solr is not available. Index fetch failed. Exception: org.apache.solr.client.solrj.SolrServerException: Server refused connection at: http://127.0.0.1:56093/solr
[junit4:junit4]   2> 81079 T378 C49 REQ [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
[junit4:junit4]   2> 81079 T373 C49 REQ [collection1] webapp=/solr path=/replication params={command=backup&numberToKeep=1} status=0 QTime=0 
[junit4:junit4]   2> 81157 T370 oash.SnapPuller.fetchLatestIndex SEVERE Master at: http://127.0.0.1:56093/solr is not available. Index fetch failed. Exception: org.apache.solr.client.solrj.SolrServerException: Server refused connection at: http://127.0.0.1:56093/solr
[junit4:junit4]   2> 81254 T375 C49 REQ [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=4 
[junit4:junit4]   2> 81303 T155 oas.SolrTestCaseJ4.tearDown ###Ending test
[junit4:junit4]   1> replicate slave to master
[junit4:junit4]   2> 81320 T155 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=23440636
[junit4:junit4]   2> 81320 T155 oasc.SolrCore.close [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@1e62c83
[junit4:junit4]   2> 81335 T155 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=1,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=493,cumulative_deletesById=0,cumulative_deletesByQuery=1,cumulative_errors=0}
[junit4:junit4]   2> 81335 T155 oasc.SolrCore.decrefSolrCoreState Closing SolrCoreState
[junit4:junit4]   2> 81335 T155 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4]   2> 81335 T155 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4:junit4]   2> 81335 T155 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4:junit4]   2> 81335 T155 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/solr,null}
[junit4:junit4]   2> 81449 T155 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=18514685
[junit4:junit4]   2> 81449 T155 oasc.SolrCore.close [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@401d45
[junit4:junit4]   2> 81449 T155 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=2,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=1,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
[junit4:junit4]   2> 81449 T155 oasc.SolrCore.decrefSolrCoreState Closing SolrCoreState
[junit4:junit4]   2> 81449 T155 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4]   2> 81449 T155 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4:junit4]   2> 81449 T155 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4:junit4]   2> 81464 T155 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/solr,null}
[junit4:junit4]   2> !!!! WARNING: best effort to remove <http://sierranevada.servebeer.com:8080/job/slow-io-beasting/ws/solr/build/solr-core/test/J5/./org.apache.solr.handler.TestReplicationHandler$SolrInstance-1351258786708/slave/collection1/data/index.20121026084039028/_7.fdt> FAILED !!!!!
[junit4:junit4]   2> !!!! WARNING: best effort to remove <http://sierranevada.servebeer.com:8080/job/slow-io-beasting/ws/solr/build/solr-core/test/J5/./org.apache.solr.handler.TestReplicationHandler$SolrInstance-1351258786708/slave/collection1/data/index.20121026084039028> FAILED !!!!!
[junit4:junit4]   2> !!!! WARNING: best effort to remove <http://sierranevada.servebeer.com:8080/job/slow-io-beasting/ws/solr/build/solr-core/test/J5/./org.apache.solr.handler.TestReplicationHandler$SolrInstance-1351258786708/slave/collection1/data> FAILED !!!!!
[junit4:junit4]   2> !!!! WARNING: best effort to remove <http://sierranevada.servebeer.com:8080/job/slow-io-beasting/ws/solr/build/solr-core/test/J5/./org.apache.solr.handler.TestReplicationHandler$SolrInstance-1351258786708/slave/collection1> FAILED !!!!!
[junit4:junit4]   2> 81727 T155 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4:junit4]   2> 82242 T370 oash.SnapPuller.fetchLatestIndex SEVERE Master at: http://127.0.0.1:56093/solr is not available. Index fetch failed. Exception: org.apache.solr.client.solrj.SolrServerException: Server refused connection at: http://127.0.0.1:56093/solr
[junit4:junit4]   2> 204387 T155 oas.SolrTestCaseJ4.endTrackingSearchers SEVERE ERROR: SolrIndexSearcher opens=61 closes=60
[junit4:junit4]   2> NOTE: test params are: codec=Lucene3x, sim=RandomSimilarityProvider(queryNorm=false,coord=crazy): {}, locale=ru_RU, timezone=America/Panama
[junit4:junit4]   2> NOTE: Windows Vista 6.0 x86/Sun Microsystems Inc. 1.6.0_23 (32-bit)/cpus=4,threads=1,free=16273944,total=58212352
[junit4:junit4]   2> NOTE: All tests run in this JVM: [ZkCLITest, TestRecovery, DirectUpdateHandlerOptimizeTest, SuggesterTSTTest, BadComponentTest, BasicFunctionalityTest, ZkControllerTest, TestLuceneMatchVersion, TestQueryTypes, SimpleFacetsTest, PreAnalyzedFieldTest, TestExtendedDismaxParser, TestReplicationHandler]
[junit4:junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestReplicationHandler -Dtests.seed=B0BEB4BFFC6640D4 -Dtests.slow=true -Dtests.locale=ru_RU -Dtests.timezone=America/Panama -Dtests.file.encoding=UTF-8
[junit4:junit4] ERROR   0.00s J5 | TestReplicationHandler (suite) <<<
[junit4:junit4]    > Throwable #1: java.lang.AssertionError: ERROR: SolrIndexSearcher opens=61 closes=60
[junit4:junit4]    > 	at __randomizedtesting.SeedInfo.seed([B0BEB4BFFC6640D4]:0)
[junit4:junit4]    > 	at org.junit.Assert.fail(Assert.java:93)
[junit4:junit4]    > 	at org.apache.solr.SolrTestCaseJ4.endTrackingSearchers(SolrTestCaseJ4.java:242)
[junit4:junit4]    > 	at org.apache.solr.SolrTestCaseJ4.afterClass(SolrTestCaseJ4.java:97)
[junit4:junit4]    > 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4:junit4]    > 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
[junit4:junit4]    > 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
[junit4:junit4]    > 	at java.lang.reflect.Method.invoke(Method.java:597)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:700)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
[junit4:junit4]    > 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
[junit4:junit4]    > 	at java.lang.Thread.run(Thread.java:662)
[junit4:junit4] Completed on J5 in 204.45s, 1 test, 1 failure <<< FAILURES!
[junit4:junit4] 
[junit4:junit4] Suite: org.apache.solr.update.processor.DefaultValueUpdateProcessorTest
[junit4:junit4] Completed on J5 in 1.47s, 1 test
[junit4:junit4] 
[junit4:junit4] Suite: org.apache.solr.schema.NotRequiredUniqueKeyTest
[junit4:junit4] Completed on J5 in 0.34s, 1 test
[junit4:junit4] 
[junit4:junit4] Suite: org.apache.solr.schema.RequiredFieldsTest
[junit4:junit4] Completed on J5 in 1.05s, 3 tests
[junit4:junit4] 
[junit4:junit4] Suite: org.apache.solr.schema.IndexSchemaTest
[junit4:junit4] Completed on J5 in 1.00s, 3 tests
[junit4:junit4] 
[junit4:junit4] 
[junit4:junit4] Tests with failures:
[junit4:junit4]   - org.apache.solr.handler.TestReplicationHandler (suite)
[junit4:junit4] 
[junit4:junit4] 
[junit4:junit4] JVM J0:     2.59 ..   272.37 =   269.78s
[junit4:junit4] JVM J1:     2.59 ..   254.87 =   252.28s
[junit4:junit4] JVM J2:     2.33 ..   268.19 =   265.87s
[junit4:junit4] JVM J3:     2.33 ..   251.69 =   249.36s
[junit4:junit4] JVM J4:     2.59 ..   278.91 =   276.32s
[junit4:junit4] JVM J5:     3.38 ..   361.07 =   357.69s
[junit4:junit4] JVM J6:     2.59 ..   276.48 =   273.89s
[junit4:junit4] JVM J7:     2.12 ..   261.53 =   259.41s
[junit4:junit4] Execution time total: 6 minutes 1 second
[junit4:junit4] Tests summary: 233 suites, 965 tests, 1 suite-level error, 13 ignored (8 assumptions)

BUILD FAILED
<http://sierranevada.servebeer.com:8080/job/slow-io-beasting/ws/build.xml>:39: The following error occurred while executing this line:
<http://sierranevada.servebeer.com:8080/job/slow-io-beasting/ws/solr/build.xml>:178: The following error occurred while executing this line:
<http://sierranevada.servebeer.com:8080/job/slow-io-beasting/ws/lucene/module-build.xml>:61: The following error occurred while executing this line:
<http://sierranevada.servebeer.com:8080/job/slow-io-beasting/ws/lucene/common-build.xml>:1171: The following error occurred while executing this line:
<http://sierranevada.servebeer.com:8080/job/slow-io-beasting/ws/lucene/common-build.xml>:839: There were test failures: 233 suites, 965 tests, 1 suite-level error, 13 ignored (8 assumptions)

Total time: 13 minutes 56 seconds
Build step 'Invoke Ant' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Build failed in Jenkins: slow-io-beasting #4870

Posted by Charlie Cron <hu...@gmail.com>.
See <http://sierranevada.servebeer.com:8080/job/slow-io-beasting/4870/>

------------------------------------------
[...truncated 14303 lines...]
[junit4:junit4]   2> 872363 T20 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:52224 which had sessionid 0x13a9d32d6480002
[junit4:junit4]   2> 872363 T40 oaz.ClientCnxn$SendThread.run Unable to read additional data from server sessionid 0x13a9d32d6480002, likely server has closed socket, closing socket connection and attempting reconnect
[junit4:junit4]   2> 872363 T20 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:52266 which had sessionid 0x13a9d32d6480003
[junit4:junit4]   2> 872363 T22 oazs.NIOServerCnxn$Factory.run NIOServerCnxn factory exited run method
[junit4:junit4]   2> 872363 T54 oaz.ClientCnxn$SendThread.run Unable to read additional data from server sessionid 0x13a9d32d6480003, likely server has closed socket, closing socket connection and attempting reconnect
[junit4:junit4]   2> 872363 T20 oazs.FinalRequestProcessor.shutdown shutdown of request processor complete
[junit4:junit4]   2> 872363 T20 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 52219
[junit4:junit4]   2> 872363 T20 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=26799663
[junit4:junit4]   2> 872363 T20 oasc.SolrCore.close [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@3ba4f1
[junit4:junit4]   2> 872363 T20 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
[junit4:junit4]   2> 872363 T20 oasc.SolrCore.decrefSolrCoreState Closing SolrCoreState
[junit4:junit4]   2> 872363 T20 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4]   2> 872363 T20 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4:junit4]   2> 872363 T20 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4:junit4]   2> 872363 T42 oasc.Overseer$ClusterStateUpdater.amILeader According to ZK I (id=88555984356114434-127.0.0.1:52219_solr-n_0000000000) am no longer a leader.
[junit4:junit4]   2> 872472 T93 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@13d0fea name:ZooKeeperConnection Watcher:127.0.0.1:52195/solr got event WatchedEvent state:Disconnected type:None path:null path:null type:None
[junit4:junit4]   2> 872472 T55 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@138ec91 name:ZooKeeperConnection Watcher:127.0.0.1:52195/solr got event WatchedEvent state:Disconnected type:None path:null path:null type:None
[junit4:junit4]   2> 872472 T93 oascc.ConnectionManager.process zkClient has disconnected
[junit4:junit4]   2> 872472 T55 oascc.ConnectionManager.process zkClient has disconnected
[junit4:junit4]   2> 872472 T67 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1e3c2c6 name:ZooKeeperConnection Watcher:127.0.0.1:52195/solr got event WatchedEvent state:Disconnected type:None path:null path:null type:None
[junit4:junit4]   2> 872472 T79 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@14c02d4 name:ZooKeeperConnection Watcher:127.0.0.1:52195/solr got event WatchedEvent state:Disconnected type:None path:null path:null type:None
[junit4:junit4]   2> 872472 T20 oaz.ZooKeeper.close Session: 0x13a9d32d6480002 closed
[junit4:junit4]   2> 872472 T41 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@17a906e name:ZooKeeperConnection Watcher:127.0.0.1:52195/solr got event WatchedEvent state:Disconnected type:None path:null path:null type:None
[junit4:junit4]   2> 872472 T41 oascc.ConnectionManager.process Client->ZooKeeper status change trigger but we are already closed
[junit4:junit4]   2> 872472 T67 oascc.ConnectionManager.process zkClient has disconnected
[junit4:junit4]   2> 872472 T79 oascc.ConnectionManager.process zkClient has disconnected
[junit4:junit4]   2> 872472 T41 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 872473 T20 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/solr,null}
[junit4:junit4]   2> 872526 T20 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 52256
[junit4:junit4]   2> 872526 T20 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=29120938
[junit4:junit4]   2> 872528 T20 oasc.SolrCore.close [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@f6852d
[junit4:junit4]   2> 872532 T20 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=1,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
[junit4:junit4]   2> 872533 T20 oasc.SolrCore.decrefSolrCoreState Closing SolrCoreState
[junit4:junit4]   2> 872533 T20 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4]   2> 872533 T20 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4:junit4]   2> 872536 T20 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4:junit4]   2> 873608 T92 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server 127.0.0.1/127.0.0.1:52195
[junit4:junit4]   2> 873623 T66 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server 127.0.0.1/127.0.0.1:52195
[junit4:junit4]   2> 874247 T78 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server 127.0.0.1/127.0.0.1:52195
[junit4:junit4]   2> 874434 T54 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server 127.0.0.1/127.0.0.1:52195
[junit4:junit4]   2> 874606 T66 oaz.ClientCnxn$SendThread.run WARNING Session 0x13a9d32d6480004 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused: no further information
[junit4:junit4]   2> 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
[junit4:junit4]   2> 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
[junit4:junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1143)
[junit4:junit4]   2> 
[junit4:junit4]   2> 874606 T92 oaz.ClientCnxn$SendThread.run WARNING Session 0x13a9d32d6480006 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused: no further information
[junit4:junit4]   2> 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
[junit4:junit4]   2> 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
[junit4:junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1143)
[junit4:junit4]   2> 
[junit4:junit4]   2> 875217 T78 oaz.ClientCnxn$SendThread.run WARNING Session 0x13a9d32d6480005 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused: no further information
[junit4:junit4]   2> 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
[junit4:junit4]   2> 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
[junit4:junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1143)
[junit4:junit4]   2> 
[junit4:junit4]   2> 875507 T55 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 875507 T20 oaz.ZooKeeper.close Session: 0x13a9d32d6480003 closed
[junit4:junit4]   2> 875517 T20 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/solr,null}
[junit4:junit4]   2> 875568 T20 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 52285
[junit4:junit4]   2> 875568 T20 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=4055775
[junit4:junit4]   2> 876288 T92 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server 127.0.0.1/127.0.0.1:52195
[junit4:junit4]   2> 876444 T66 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server 127.0.0.1/127.0.0.1:52195
[junit4:junit4]   2> 876694 T78 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server 127.0.0.1/127.0.0.1:52195
[junit4:junit4]   2> 877287 T92 oaz.ClientCnxn$SendThread.run WARNING Session 0x13a9d32d6480006 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused: no further information
[junit4:junit4]   2> 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
[junit4:junit4]   2> 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
[junit4:junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1143)
[junit4:junit4]   2> 
[junit4:junit4]   2> 877554 T67 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 877554 T20 oaz.ZooKeeper.close Session: 0x13a9d32d6480004 closed
[junit4:junit4]   2> 877569 T20 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/solr,null}
[junit4:junit4]   2> 877620 T20 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 53072
[junit4:junit4]   2> 877620 T20 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=14194572
[junit4:junit4]   2> 877620 T20 oasc.SolrCore.close [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@6745b9
[junit4:junit4]   2> 877652 T20 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=1,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
[junit4:junit4]   2> 877652 T20 oasc.SolrCore.decrefSolrCoreState Closing SolrCoreState
[junit4:junit4]   2> 877652 T20 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4]   2> 877652 T20 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4:junit4]   2> 877652 T20 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4:junit4]   2> 877808 T79 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 877808 T20 oaz.ZooKeeper.close Session: 0x13a9d32d6480005 closed
[junit4:junit4]   2> 877823 T20 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/solr,null}
[junit4:junit4]   2> 877874 T20 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 53079
[junit4:junit4]   2> 877874 T20 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=1056169
[junit4:junit4]   2> 877874 T20 oasc.SolrCore.close [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@146ac5a
[junit4:junit4]   2> 877905 T20 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
[junit4:junit4]   2> 877905 T20 oasc.SolrCore.decrefSolrCoreState Closing SolrCoreState
[junit4:junit4]   2> 877905 T20 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4]   2> 877905 T20 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4:junit4]   2> 877905 T20 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4:junit4]   2> 878436 T92 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server 127.0.0.1/127.0.0.1:52195
[junit4:junit4]   2> 879543 T93 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 879543 T20 oaz.ZooKeeper.close Session: 0x13a9d32d6480006 closed
[junit4:junit4]   2> 879559 T20 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/solr,null}
[junit4:junit4]   2> 879813 T20 oas.SolrTestCaseJ4.tearDown ###Ending testDistribSearch
[junit4:junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=BasicDistributedZkTest -Dtests.method=testDistribSearch -Dtests.seed=5931ACB8E3018C7A -Dtests.slow=true -Dtests.locale=es_PR -Dtests.timezone=Europe/Guernsey -Dtests.file.encoding=Cp1252
[junit4:junit4] FAILURE  880s J0 | BasicDistributedZkTest.testDistribSearch <<<
[junit4:junit4]    > Throwable #1: java.lang.AssertionError: There are still nodes recoverying
[junit4:junit4]    > 	at __randomizedtesting.SeedInfo.seed([5931ACB8E3018C7A:D8D722A0945EEC46]:0)
[junit4:junit4]    > 	at org.junit.Assert.fail(Assert.java:93)
[junit4:junit4]    > 	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:154)
[junit4:junit4]    > 	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:115)
[junit4:junit4]    > 	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:110)
[junit4:junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.waitForRecoveriesToFinish(AbstractFullDistribZkTestBase.java:553)
[junit4:junit4]    > 	at org.apache.solr.cloud.BasicDistributedZkTest.doTest(BasicDistributedZkTest.java:162)
[junit4:junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:695)
[junit4:junit4]    > 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4:junit4]    > 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
[junit4:junit4]    > 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
[junit4:junit4]    > 	at java.lang.reflect.Method.invoke(Method.java:597)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
[junit4:junit4]    > 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
[junit4:junit4]    > 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
[junit4:junit4]    > 	at java.lang.Thread.run(Thread.java:662)
[junit4:junit4]   2> 879860 T20 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4:junit4]   2> 1002554 T20 oas.SolrTestCaseJ4.endTrackingSearchers SEVERE ERROR: SolrIndexSearcher opens=7 closes=6
[junit4:junit4]   2> 1002554 T19 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 1 leaked thread(s).
[junit4:junit4]   2> 1022709 T19 ccr.ThreadLeakControl.checkThreadLeaks SEVERE 1 thread leaked from SUITE scope at org.apache.solr.cloud.BasicDistributedZkTest: 
[junit4:junit4]   2> 	   1) Thread[id=68, name=searcherExecutor-40-thread-1, state=WAITING, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
[junit4:junit4]   2> 	        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:662)
[junit4:junit4]   2> 1022709 T19 ccr.ThreadLeakControl.tryToInterruptAll Starting to interrupt leaked threads:
[junit4:junit4]   2> 	   1) Thread[id=68, name=searcherExecutor-40-thread-1, state=WAITING, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]   2> 1025798 T19 ccr.ThreadLeakControl.tryToInterruptAll SEVERE There are still zombie threads that couldn't be terminated:
[junit4:junit4]   2> 	   1) Thread[id=68, name=searcherExecutor-40-thread-1, state=WAITING, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
[junit4:junit4]   2> 	        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:662)
[junit4:junit4]   2> NOTE: test params are: codec=Compressing(storedFieldsFormat=CompressingStoredFieldsFormat(compressionMode=HIGH_COMPRESSION, chunkSize=28, storedFieldsIndex=MEMORY_CHUNK)), sim=RandomSimilarityProvider(queryNorm=true,coord=yes): {}, locale=es_PR, timezone=Europe/Guernsey
[junit4:junit4]   2> NOTE: Windows Vista 6.0 x86/Sun Microsystems Inc. 1.6.0_23 (32-bit)/cpus=4,threads=2,free=4449480,total=16449536
[junit4:junit4]   2> NOTE: All tests run in this JVM: [PathHierarchyTokenizerFactoryTest, SimplePostToolTest, CoreContainerCoreInitFailuresTest, BasicDistributedZkTest]
[junit4:junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=BasicDistributedZkTest -Dtests.seed=5931ACB8E3018C7A -Dtests.slow=true -Dtests.locale=es_PR -Dtests.timezone=Europe/Guernsey -Dtests.file.encoding=Cp1252
[junit4:junit4] ERROR   0.00s J0 | BasicDistributedZkTest (suite) <<<
[junit4:junit4]    > Throwable #1: java.lang.AssertionError: ERROR: SolrIndexSearcher opens=7 closes=6
[junit4:junit4]    > 	at __randomizedtesting.SeedInfo.seed([5931ACB8E3018C7A]:0)
[junit4:junit4]    > 	at org.junit.Assert.fail(Assert.java:93)
[junit4:junit4]    > 	at org.apache.solr.SolrTestCaseJ4.endTrackingSearchers(SolrTestCaseJ4.java:242)
[junit4:junit4]    > 	at org.apache.solr.SolrTestCaseJ4.afterClass(SolrTestCaseJ4.java:97)
[junit4:junit4]    > 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4:junit4]    > 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
[junit4:junit4]    > 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
[junit4:junit4]    > 	at java.lang.reflect.Method.invoke(Method.java:597)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:700)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
[junit4:junit4]    > 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
[junit4:junit4]    > 	at java.lang.Thread.run(Thread.java:662)
[junit4:junit4]    > Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.cloud.BasicDistributedZkTest: 
[junit4:junit4]    >    1) Thread[id=68, name=searcherExecutor-40-thread-1, state=WAITING, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]    >         at sun.misc.Unsafe.park(Native Method)
[junit4:junit4]    >         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
[junit4:junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
[junit4:junit4]    >         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
[junit4:junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
[junit4:junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:662)
[junit4:junit4]    > 	at __randomizedtesting.SeedInfo.seed([5931ACB8E3018C7A]:0)
[junit4:junit4]    > Throwable #3: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
[junit4:junit4]    >    1) Thread[id=68, name=searcherExecutor-40-thread-1, state=WAITING, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]    >         at sun.misc.Unsafe.park(Native Method)
[junit4:junit4]    >         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
[junit4:junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
[junit4:junit4]    >         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
[junit4:junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
[junit4:junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:662)
[junit4:junit4]    > 	at __randomizedtesting.SeedInfo.seed([5931ACB8E3018C7A]:0)
[junit4:junit4] Completed on J0 in 1025.86s, 1 test, 2 failures, 2 errors <<< FAILURES!
[junit4:junit4] 
[junit4:junit4] 
[junit4:junit4] Tests with failures:
[junit4:junit4]   - org.apache.solr.cloud.BasicDistributedZkTest.testDistribSearch
[junit4:junit4]   - org.apache.solr.cloud.BasicDistributedZkTest (suite)
[junit4:junit4] 
[junit4:junit4] 
[junit4:junit4] JVM J0:     2.59 ..  1061.41 =  1058.82s
[junit4:junit4] JVM J1:     2.36 ..   212.27 =   209.91s
[junit4:junit4] JVM J2:     2.59 ..   225.12 =   222.53s
[junit4:junit4] JVM J3:     2.59 ..   281.84 =   279.25s
[junit4:junit4] JVM J4:     2.59 ..   227.87 =   225.28s
[junit4:junit4] JVM J5:     2.59 ..   214.53 =   211.94s
[junit4:junit4] JVM J6:     2.59 ..   228.34 =   225.75s
[junit4:junit4] JVM J7:     2.59 ..   213.99 =   211.40s
[junit4:junit4] Execution time total: 17 minutes 41 seconds
[junit4:junit4] Tests summary: 233 suites, 965 tests, 3 suite-level errors, 1 failure, 12 ignored (7 assumptions)

BUILD FAILED
<http://sierranevada.servebeer.com:8080/job/slow-io-beasting/ws/build.xml>:39: The following error occurred while executing this line:
<http://sierranevada.servebeer.com:8080/job/slow-io-beasting/ws/solr/build.xml>:178: The following error occurred while executing this line:
<http://sierranevada.servebeer.com:8080/job/slow-io-beasting/ws/lucene/module-build.xml>:61: The following error occurred while executing this line:
<http://sierranevada.servebeer.com:8080/job/slow-io-beasting/ws/lucene/common-build.xml>:1171: The following error occurred while executing this line:
<http://sierranevada.servebeer.com:8080/job/slow-io-beasting/ws/lucene/common-build.xml>:839: There were test failures: 233 suites, 965 tests, 3 suite-level errors, 1 failure, 12 ignored (7 assumptions)

Total time: 24 minutes 57 seconds
Build step 'Invoke Ant' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org