You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "selckin (JIRA)" <ji...@apache.org> on 2011/06/19 13:20:47 UTC
[jira] [Created] (SOLR-2608) TestReplicationHandler is flakey
TestReplicationHandler is flakey
--------------------------------
Key: SOLR-2608
URL: https://issues.apache.org/jira/browse/SOLR-2608
Project: Solr
Issue Type: Bug
Reporter: selckin
I've been running some while(1) tests on trunk, and TestReplicationHandler is very flakey it fails about every 10th run.
Probably not a bug, but the test not waiting correctly
{code}
[junit] Testsuite: org.apache.solr.handler.TestReplicationHandler
[junit] Testcase: org.apache.solr.handler.TestReplicationHandler: FAILED
[junit] ERROR: SolrIndexSearcher opens=48 closes=47
[junit] junit.framework.AssertionFailedError: ERROR: SolrIndexSearcher opens=48 closes=47
[junit] at org.apache.solr.SolrTestCaseJ4.endTrackingSearchers(SolrTestCaseJ4.java:131)
[junit] at org.apache.solr.SolrTestCaseJ4.afterClassSolrTestCase(SolrTestCaseJ4.java:74)
[junit]
[junit]
[junit] Tests run: 8, Failures: 1, Errors: 0, Time elapsed: 40.772 sec
[junit]
[junit] ------------- Standard Error -----------------
[junit] 19-Jun-2011 21:26:44 org.apache.solr.handler.SnapPuller fetchLatestIndex
[junit] SEVERE: Master at: http://localhost:51817/solr/replication is not available. Index fetch failed. Exception: Connection refused
[junit] 19-Jun-2011 21:26:49 org.apache.solr.common.SolrException log
[junit] SEVERE: java.util.concurrent.RejectedExecutionException
[junit] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:1768)
[junit] at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:767)
[junit] at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:658)
[junit] at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:92)
[junit] at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:603)
[junit] at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1149)
[junit] at org.apache.solr.update.DirectUpdateHandler2.commit(DirectUpdateHandler2.java:346)
[junit] at org.apache.solr.handler.SnapPuller.doCommit(SnapPuller.java:483)
[junit] at org.apache.solr.handler.SnapPuller.fetchLatestIndex(SnapPuller.java:332)
[junit] at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:267)
[junit] at org.apache.solr.handler.SnapPuller$1.run(SnapPuller.java:166)
[junit] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
[junit] at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
[junit] at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
[junit] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
[junit] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:180)
[junit] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:204)
[junit] at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
[junit] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
[junit] at java.lang.Thread.run(Thread.java:662)
[junit]
[junit] 19-Jun-2011 21:26:51 org.apache.solr.update.SolrIndexWriter finalize
[junit] SEVERE: SolrIndexWriter was not closed prior to finalize(), indicates a bug -- POSSIBLE RESOURCE LEAK!!!
[junit] 19-Jun-2011 21:26:51 org.apache.solr.common.util.ConcurrentLRUCache finalize
[junit] SEVERE: ConcurrentLRUCache was not destroyed prior to finalize(), indicates a bug -- POSSIBLE RESOURCE LEAK!!!
[junit] 19-Jun-2011 21:27:13 org.apache.solr.handler.SnapPuller fetchLatestIndex
[junit] SEVERE: Master at: http://localhost:46559/solr/replication is not available. Index fetch failed. Exception: Connection refused
[junit] 19-Jun-2011 21:27:14 org.apache.solr.handler.SnapPuller fetchLatestIndex
[junit] SEVERE: Master at: http://localhost:46559/solr/replication is not available. Index fetch failed. Exception: Connection refused
[junit] 19-Jun-2011 21:27:14 org.apache.solr.SolrTestCaseJ4 endTrackingSearchers
[junit] SEVERE: ERROR: SolrIndexSearcher opens=48 closes=47
[junit] ------------- ---------------- ---------------
[junit] TEST org.apache.solr.handler.TestReplicationHandler FAILED
{code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (SOLR-2608) TestReplicationHandler is flakey
Posted by "Hoss Man (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13454243#comment-13454243 ]
Hoss Man commented on SOLR-2608:
--------------------------------
I can't comment on the specific exceptions mentioned above, but havin recently looked at TestReplicationHandler because of SOLR-3809 i noticed a few things i thought i'd comment on here...
at some point it was annotated as @Slow - i believe the crux of the problem with why it can be very slow for some people is that the majority of the functionality being tested seems to rely on the slave polling the master for replication, and the "rQuery" method used through out the test will retry queries over and over (up to 30 seconds) until they pass.
While we should definitley have some test that the polling works, a lot of the functionality not-polling specific could probably be tested more reliably using "on demand" snappulling commands to the slave.
> TestReplicationHandler is flakey
> --------------------------------
>
> Key: SOLR-2608
> URL: https://issues.apache.org/jira/browse/SOLR-2608
> Project: Solr
> Issue Type: Bug
> Reporter: selckin
>
> I've been running some while(1) tests on trunk, and TestReplicationHandler is very flakey it fails about every 10th run.
> Probably not a bug, but the test not waiting correctly
> {code}
> [junit] Testsuite: org.apache.solr.handler.TestReplicationHandler
> [junit] Testcase: org.apache.solr.handler.TestReplicationHandler: FAILED
> [junit] ERROR: SolrIndexSearcher opens=48 closes=47
> [junit] junit.framework.AssertionFailedError: ERROR: SolrIndexSearcher opens=48 closes=47
> [junit] at org.apache.solr.SolrTestCaseJ4.endTrackingSearchers(SolrTestCaseJ4.java:131)
> [junit] at org.apache.solr.SolrTestCaseJ4.afterClassSolrTestCase(SolrTestCaseJ4.java:74)
> [junit]
> [junit]
> [junit] Tests run: 8, Failures: 1, Errors: 0, Time elapsed: 40.772 sec
> [junit]
> [junit] ------------- Standard Error -----------------
> [junit] 19-Jun-2011 21:26:44 org.apache.solr.handler.SnapPuller fetchLatestIndex
> [junit] SEVERE: Master at: http://localhost:51817/solr/replication is not available. Index fetch failed. Exception: Connection refused
> [junit] 19-Jun-2011 21:26:49 org.apache.solr.common.SolrException log
> [junit] SEVERE: java.util.concurrent.RejectedExecutionException
> [junit] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:1768)
> [junit] at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:767)
> [junit] at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:658)
> [junit] at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:92)
> [junit] at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:603)
> [junit] at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1149)
> [junit] at org.apache.solr.update.DirectUpdateHandler2.commit(DirectUpdateHandler2.java:346)
> [junit] at org.apache.solr.handler.SnapPuller.doCommit(SnapPuller.java:483)
> [junit] at org.apache.solr.handler.SnapPuller.fetchLatestIndex(SnapPuller.java:332)
> [junit] at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:267)
> [junit] at org.apache.solr.handler.SnapPuller$1.run(SnapPuller.java:166)
> [junit] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
> [junit] at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
> [junit] at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
> [junit] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
> [junit] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:180)
> [junit] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:204)
> [junit] at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> [junit] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> [junit] at java.lang.Thread.run(Thread.java:662)
> [junit]
> [junit] 19-Jun-2011 21:26:51 org.apache.solr.update.SolrIndexWriter finalize
> [junit] SEVERE: SolrIndexWriter was not closed prior to finalize(), indicates a bug -- POSSIBLE RESOURCE LEAK!!!
> [junit] 19-Jun-2011 21:26:51 org.apache.solr.common.util.ConcurrentLRUCache finalize
> [junit] SEVERE: ConcurrentLRUCache was not destroyed prior to finalize(), indicates a bug -- POSSIBLE RESOURCE LEAK!!!
> [junit] 19-Jun-2011 21:27:13 org.apache.solr.handler.SnapPuller fetchLatestIndex
> [junit] SEVERE: Master at: http://localhost:46559/solr/replication is not available. Index fetch failed. Exception: Connection refused
> [junit] 19-Jun-2011 21:27:14 org.apache.solr.handler.SnapPuller fetchLatestIndex
> [junit] SEVERE: Master at: http://localhost:46559/solr/replication is not available. Index fetch failed. Exception: Connection refused
> [junit] 19-Jun-2011 21:27:14 org.apache.solr.SolrTestCaseJ4 endTrackingSearchers
> [junit] SEVERE: ERROR: SolrIndexSearcher opens=48 closes=47
> [junit] ------------- ---------------- ---------------
> [junit] TEST org.apache.solr.handler.TestReplicationHandler FAILED
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (SOLR-2608) TestReplicationHandler is flakey
Posted by "Sami Siren (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13219909#comment-13219909 ]
Sami Siren commented on SOLR-2608:
----------------------------------
I am also seeing this test fail quite often. The stacktrace is now different:
{code}
23987 T1101 oasc.SolrException.log SEVERE SnapPull failed :org.apache.solr.common.SolrException
at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1388)
at org.apache.solr.handler.SnapPuller.doCommit(SnapPuller.java:505)
at org.apache.solr.handler.SnapPuller.fetchLatestIndex(SnapPuller.java:348)
at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:298)
at org.apache.solr.handler.SnapPuller$1.run(SnapPuller.java:163)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:204)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.util.concurrent.RejectedExecutionException
at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:1768)
at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:767)
at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:658)
at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:92)
at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:603)
at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1356)
... 13 more
25051 T3 oasu.ConcurrentLRUCache.finalize SEVERE ConcurrentLRUCache was not destroyed prior to finalize(), indicates a bug -- POSSIBLE RESOURCE LEAK!!!
40748 T1120 oasc.SolrException.log SEVERE SnapPull failed :org.apache.solr.common.SolrException: Unable to download _7_1.del completely. Downloaded 0!=92
at org.apache.solr.handler.SnapPuller$FileFetcher.cleanup(SnapPuller.java:1081)
at org.apache.solr.handler.SnapPuller$FileFetcher.fetchFile(SnapPuller.java:961)
at org.apache.solr.handler.SnapPuller.downloadIndexFiles(SnapPuller.java:587)
at org.apache.solr.handler.SnapPuller.fetchLatestIndex(SnapPuller.java:322)
at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:298)
at org.apache.solr.handler.ReplicationHandler$1.run(ReplicationHandler.java:179)
{code}
> TestReplicationHandler is flakey
> --------------------------------
>
> Key: SOLR-2608
> URL: https://issues.apache.org/jira/browse/SOLR-2608
> Project: Solr
> Issue Type: Bug
> Reporter: selckin
>
> I've been running some while(1) tests on trunk, and TestReplicationHandler is very flakey it fails about every 10th run.
> Probably not a bug, but the test not waiting correctly
> {code}
> [junit] Testsuite: org.apache.solr.handler.TestReplicationHandler
> [junit] Testcase: org.apache.solr.handler.TestReplicationHandler: FAILED
> [junit] ERROR: SolrIndexSearcher opens=48 closes=47
> [junit] junit.framework.AssertionFailedError: ERROR: SolrIndexSearcher opens=48 closes=47
> [junit] at org.apache.solr.SolrTestCaseJ4.endTrackingSearchers(SolrTestCaseJ4.java:131)
> [junit] at org.apache.solr.SolrTestCaseJ4.afterClassSolrTestCase(SolrTestCaseJ4.java:74)
> [junit]
> [junit]
> [junit] Tests run: 8, Failures: 1, Errors: 0, Time elapsed: 40.772 sec
> [junit]
> [junit] ------------- Standard Error -----------------
> [junit] 19-Jun-2011 21:26:44 org.apache.solr.handler.SnapPuller fetchLatestIndex
> [junit] SEVERE: Master at: http://localhost:51817/solr/replication is not available. Index fetch failed. Exception: Connection refused
> [junit] 19-Jun-2011 21:26:49 org.apache.solr.common.SolrException log
> [junit] SEVERE: java.util.concurrent.RejectedExecutionException
> [junit] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:1768)
> [junit] at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:767)
> [junit] at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:658)
> [junit] at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:92)
> [junit] at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:603)
> [junit] at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1149)
> [junit] at org.apache.solr.update.DirectUpdateHandler2.commit(DirectUpdateHandler2.java:346)
> [junit] at org.apache.solr.handler.SnapPuller.doCommit(SnapPuller.java:483)
> [junit] at org.apache.solr.handler.SnapPuller.fetchLatestIndex(SnapPuller.java:332)
> [junit] at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:267)
> [junit] at org.apache.solr.handler.SnapPuller$1.run(SnapPuller.java:166)
> [junit] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
> [junit] at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
> [junit] at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
> [junit] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
> [junit] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:180)
> [junit] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:204)
> [junit] at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> [junit] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> [junit] at java.lang.Thread.run(Thread.java:662)
> [junit]
> [junit] 19-Jun-2011 21:26:51 org.apache.solr.update.SolrIndexWriter finalize
> [junit] SEVERE: SolrIndexWriter was not closed prior to finalize(), indicates a bug -- POSSIBLE RESOURCE LEAK!!!
> [junit] 19-Jun-2011 21:26:51 org.apache.solr.common.util.ConcurrentLRUCache finalize
> [junit] SEVERE: ConcurrentLRUCache was not destroyed prior to finalize(), indicates a bug -- POSSIBLE RESOURCE LEAK!!!
> [junit] 19-Jun-2011 21:27:13 org.apache.solr.handler.SnapPuller fetchLatestIndex
> [junit] SEVERE: Master at: http://localhost:46559/solr/replication is not available. Index fetch failed. Exception: Connection refused
> [junit] 19-Jun-2011 21:27:14 org.apache.solr.handler.SnapPuller fetchLatestIndex
> [junit] SEVERE: Master at: http://localhost:46559/solr/replication is not available. Index fetch failed. Exception: Connection refused
> [junit] 19-Jun-2011 21:27:14 org.apache.solr.SolrTestCaseJ4 endTrackingSearchers
> [junit] SEVERE: ERROR: SolrIndexSearcher opens=48 closes=47
> [junit] ------------- ---------------- ---------------
> [junit] TEST org.apache.solr.handler.TestReplicationHandler FAILED
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org