You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Chris Hostetter <ho...@fucit.org> on 2015/05/23 05:29:29 UTC
why didn't the test timeout? -- was: Re: [jira] [Updated] (SOLR-7587)
TestSpellCheckResponse stalled and never timed out -- possible VersionBucket
bug? (5.2 branch)
Dawid: seperte from the questions raised in Jira about hte underlying
problem in Solr, any ideas why the framework didn't time this test out
long before the 110 minute mark when i noticed it still running?
(I don't see anything in the test or it's baseclass overriding hte default
timeouts.)
: Date: Sat, 23 May 2015 03:23:17 +0000 (UTC)
: From: "Hoss Man (JIRA)" <ji...@apache.org>
: Reply-To: dev@lucene.apache.org
: To: dev@lucene.apache.org
: Subject: [jira] [Updated] (SOLR-7587) TestSpellCheckResponse stalled and never
: timed out -- possible VersionBucket bug? (5.2 branch)
:
:
: [ https://issues.apache.org/jira/browse/SOLR-7587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
:
: Hoss Man updated SOLR-7587:
: ---------------------------
: Attachment: jstack.1.txt
: jstack.2.txt
: junit4-J0-20150522_181244_599.events
: junit4-J0-20150522_181244_599.spill
: junit4-J0-20150522_181244_599.suites
:
: 2 thread dumps, and the non-empty J0 files from solr/build/solr-solrj/test/temp/
:
: Most interesting looking thread...
:
: {noformat}
: "TEST-TestSpellCheckResponse.testSpellCheckResponse-seed#[FA0A9DF72EDC5BCD]" prio=10 tid=0x00007f10843da000 nid=0x2ff9 waiting on condition [0x00007f10c10f1000]
: java.lang.Thread.State: WAITING (parking)
: at sun.misc.Unsafe.park(Native Method)
: - parking to wait for <0x00000000f7f383e0> (a java.util.concurrent.locks.ReentrantReadWriteLock$FairSync)
: at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
: at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:834)
: at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:867)
: at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1197)
: at java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.lock(ReentrantReadWriteLock.java:945)
: at org.apache.solr.update.VersionInfo.blockUpdates(VersionInfo.java:118)
: at org.apache.solr.update.UpdateLog.onFirstSearcher(UpdateLog.java:1604)
: at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1810)
: at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1505)
: at org.apache.solr.update.DirectUpdateHandler2.commit(DirectUpdateHandler2.java:617)
: - locked <0x00000000f6f09a10> (a java.lang.Object)
: at org.apache.solr.update.processor.RunUpdateProcessor.processCommit(RunUpdateProcessorFactory.java:95)
: at org.apache.solr.update.processor.UpdateRequestProcessor.processCommit(UpdateRequestProcessor.java:64)
: at org.apache.solr.update.processor.DistributedUpdateProcessor.doLocalCommit(DistributedUpdateProcessor.java:1635)
: at org.apache.solr.update.processor.DistributedUpdateProcessor.processCommit(DistributedUpdateProcessor.java:1612)
: at org.apache.solr.update.processor.LogUpdateProcessor.processCommit(LogUpdateProcessorFactory.java:161)
: at org.apache.solr.handler.RequestHandlerUtils.handleCommit(RequestHandlerUtils.java:69)
: at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:68)
: at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:143)
: at org.apache.solr.core.SolrCore.execute(SolrCore.java:2051)
: at org.apache.solr.client.solrj.embedded.EmbeddedSolrServer.request(EmbeddedSolrServer.java:179)
: at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:135)
: at org.apache.solr.client.solrj.SolrClient.commit(SolrClient.java:483)
: at org.apache.solr.client.solrj.SolrClient.commit(SolrClient.java:502)
: at org.apache.solr.client.solrj.response.TestSpellCheckResponse.testSpellCheckResponse(TestSpellCheckResponse.java:51)
: {noformat}
:
:
:
: > TestSpellCheckResponse stalled and never timed out -- possible VersionBucket bug? (5.2 branch)
: > ----------------------------------------------------------------------------------------------
: >
: > Key: SOLR-7587
: > URL: https://issues.apache.org/jira/browse/SOLR-7587
: > Project: Solr
: > Issue Type: Bug
: > Reporter: Hoss Man
: > Attachments: jstack.1.txt, jstack.2.txt, junit4-J0-20150522_181244_599.events, junit4-J0-20150522_181244_599.spill, junit4-J0-20150522_181244_599.suites
: >
: >
: > On the 5.2 branch (r1681250), I encountered a solrj test stalled for over 110 minutes before i finally killed it...
: > {noformat}
: > [junit4] Suite: org.apache.solr.common.util.TestRetryUtil
: > [junit4] Completed [55/60] on J1 in 1.04s, 1 test
: > [junit4]
: > [junit4] HEARTBEAT J0 PID(12147@tray): 2015-05-22T18:14:56, stalled for 121s at: TestSpellCheckResponse.testSpellCheckResponse
: > [junit4] HEARTBEAT J0 PID(12147@tray): 2015-05-22T18:15:56, stalled for 181s at: TestSpellCheckResponse.testSpellCheckResponse
: > ...
: > [junit4] HEARTBEAT J0 PID(12147@tray): 2015-05-22T20:00:56, stalled for 6481s at: TestSpellCheckResponse.testSpellCheckResponse
: > [junit4] HEARTBEAT J0 PID(12147@tray): 2015-05-22T20:01:56, stalled for 6541s at: TestSpellCheckResponse.testSpellCheckResponse
: > [junit4] HEARTBEAT J0 PID(12147@tray): 2015-05-22T20:02:56, stalled for 6601s at: TestSpellCheckResponse.testSpellCheckResponse
: > {noformat}
: > I'll attach some jstack output as well as all the temp files from the J0 runner.
:
:
:
: --
: This message was sent by Atlassian JIRA
: (v6.3.4#6332)
:
: ---------------------------------------------------------------------
: To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
: For additional commands, e-mail: dev-help@lucene.apache.org
:
:
-Hoss
http://www.lucidworks.com/
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: why didn't the test timeout? -- was: Re: [jira] [Updated]
(SOLR-7587) TestSpellCheckResponse stalled and never timed out -- possible
VersionBucket bug? (5.2 branch)
Posted by Dawid Weiss <da...@cs.put.poznan.pl>.
No problem at all. I think it might have been 60 minutes initially but
nightly tests (or some bad combinations of components) didn't complete
within that limit. Perhaps it's time to revise this and lower the
worst case/ add overrides where really applicable.
Dawid
On Tue, May 26, 2015 at 6:32 PM, Chris Hostetter
<ho...@fucit.org> wrote:
>
> : The default timeout seems to be 7200000 millis, this means 7200
> : seconds or ~120 minutes. Look for @TimeoutSuite annotation in the
>
> thanks ... my bad -- i did look for TimeoutSuite in the test, but i
> thought the default was 60 minutes. (forgot to double check that
> assumption)
>
> false alarm (on the timeout)
>
>
> -Hoss
> http://www.lucidworks.com/
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: why didn't the test timeout? -- was: Re: [jira] [Updated]
(SOLR-7587) TestSpellCheckResponse stalled and never timed out -- possible
VersionBucket bug? (5.2 branch)
Posted by Chris Hostetter <ho...@fucit.org>.
: The default timeout seems to be 7200000 millis, this means 7200
: seconds or ~120 minutes. Look for @TimeoutSuite annotation in the
thanks ... my bad -- i did look for TimeoutSuite in the test, but i
thought the default was 60 minutes. (forgot to double check that
assumption)
false alarm (on the timeout)
-Hoss
http://www.lucidworks.com/
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: why didn't the test timeout? -- was: Re: [jira] [Updated]
(SOLR-7587) TestSpellCheckResponse stalled and never timed out -- possible
VersionBucket bug? (5.2 branch)
Posted by Dawid Weiss <da...@cs.put.poznan.pl>.
The default timeout seems to be 7200000 millis, this means 7200
seconds or ~120 minutes. Look for @TimeoutSuite annotation in the
code, it can be overridden. Unfortunately there is currently no way to
pick a value dynamically (for example to vary nightly tests against
regular runs); this is Java annotation system constraint rather than
anything else (no dynamic code in annotations).
Dawid
On Sat, May 23, 2015 at 5:29 AM, Chris Hostetter
<ho...@fucit.org> wrote:
>
> Dawid: seperte from the questions raised in Jira about hte underlying
> problem in Solr, any ideas why the framework didn't time this test out
> long before the 110 minute mark when i noticed it still running?
>
> (I don't see anything in the test or it's baseclass overriding hte default
> timeouts.)
>
>
> : Date: Sat, 23 May 2015 03:23:17 +0000 (UTC)
> : From: "Hoss Man (JIRA)" <ji...@apache.org>
> : Reply-To: dev@lucene.apache.org
> : To: dev@lucene.apache.org
> : Subject: [jira] [Updated] (SOLR-7587) TestSpellCheckResponse stalled and never
> : timed out -- possible VersionBucket bug? (5.2 branch)
> :
> :
> : [ https://issues.apache.org/jira/browse/SOLR-7587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
> :
> : Hoss Man updated SOLR-7587:
> : ---------------------------
> : Attachment: jstack.1.txt
> : jstack.2.txt
> : junit4-J0-20150522_181244_599.events
> : junit4-J0-20150522_181244_599.spill
> : junit4-J0-20150522_181244_599.suites
> :
> : 2 thread dumps, and the non-empty J0 files from solr/build/solr-solrj/test/temp/
> :
> : Most interesting looking thread...
> :
> : {noformat}
> : "TEST-TestSpellCheckResponse.testSpellCheckResponse-seed#[FA0A9DF72EDC5BCD]" prio=10 tid=0x00007f10843da000 nid=0x2ff9 waiting on condition [0x00007f10c10f1000]
> : java.lang.Thread.State: WAITING (parking)
> : at sun.misc.Unsafe.park(Native Method)
> : - parking to wait for <0x00000000f7f383e0> (a java.util.concurrent.locks.ReentrantReadWriteLock$FairSync)
> : at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
> : at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:834)
> : at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:867)
> : at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1197)
> : at java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.lock(ReentrantReadWriteLock.java:945)
> : at org.apache.solr.update.VersionInfo.blockUpdates(VersionInfo.java:118)
> : at org.apache.solr.update.UpdateLog.onFirstSearcher(UpdateLog.java:1604)
> : at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1810)
> : at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1505)
> : at org.apache.solr.update.DirectUpdateHandler2.commit(DirectUpdateHandler2.java:617)
> : - locked <0x00000000f6f09a10> (a java.lang.Object)
> : at org.apache.solr.update.processor.RunUpdateProcessor.processCommit(RunUpdateProcessorFactory.java:95)
> : at org.apache.solr.update.processor.UpdateRequestProcessor.processCommit(UpdateRequestProcessor.java:64)
> : at org.apache.solr.update.processor.DistributedUpdateProcessor.doLocalCommit(DistributedUpdateProcessor.java:1635)
> : at org.apache.solr.update.processor.DistributedUpdateProcessor.processCommit(DistributedUpdateProcessor.java:1612)
> : at org.apache.solr.update.processor.LogUpdateProcessor.processCommit(LogUpdateProcessorFactory.java:161)
> : at org.apache.solr.handler.RequestHandlerUtils.handleCommit(RequestHandlerUtils.java:69)
> : at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:68)
> : at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:143)
> : at org.apache.solr.core.SolrCore.execute(SolrCore.java:2051)
> : at org.apache.solr.client.solrj.embedded.EmbeddedSolrServer.request(EmbeddedSolrServer.java:179)
> : at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:135)
> : at org.apache.solr.client.solrj.SolrClient.commit(SolrClient.java:483)
> : at org.apache.solr.client.solrj.SolrClient.commit(SolrClient.java:502)
> : at org.apache.solr.client.solrj.response.TestSpellCheckResponse.testSpellCheckResponse(TestSpellCheckResponse.java:51)
> : {noformat}
> :
> :
> :
> : > TestSpellCheckResponse stalled and never timed out -- possible VersionBucket bug? (5.2 branch)
> : > ----------------------------------------------------------------------------------------------
> : >
> : > Key: SOLR-7587
> : > URL: https://issues.apache.org/jira/browse/SOLR-7587
> : > Project: Solr
> : > Issue Type: Bug
> : > Reporter: Hoss Man
> : > Attachments: jstack.1.txt, jstack.2.txt, junit4-J0-20150522_181244_599.events, junit4-J0-20150522_181244_599.spill, junit4-J0-20150522_181244_599.suites
> : >
> : >
> : > On the 5.2 branch (r1681250), I encountered a solrj test stalled for over 110 minutes before i finally killed it...
> : > {noformat}
> : > [junit4] Suite: org.apache.solr.common.util.TestRetryUtil
> : > [junit4] Completed [55/60] on J1 in 1.04s, 1 test
> : > [junit4]
> : > [junit4] HEARTBEAT J0 PID(12147@tray): 2015-05-22T18:14:56, stalled for 121s at: TestSpellCheckResponse.testSpellCheckResponse
> : > [junit4] HEARTBEAT J0 PID(12147@tray): 2015-05-22T18:15:56, stalled for 181s at: TestSpellCheckResponse.testSpellCheckResponse
> : > ...
> : > [junit4] HEARTBEAT J0 PID(12147@tray): 2015-05-22T20:00:56, stalled for 6481s at: TestSpellCheckResponse.testSpellCheckResponse
> : > [junit4] HEARTBEAT J0 PID(12147@tray): 2015-05-22T20:01:56, stalled for 6541s at: TestSpellCheckResponse.testSpellCheckResponse
> : > [junit4] HEARTBEAT J0 PID(12147@tray): 2015-05-22T20:02:56, stalled for 6601s at: TestSpellCheckResponse.testSpellCheckResponse
> : > {noformat}
> : > I'll attach some jstack output as well as all the temp files from the J0 runner.
> :
> :
> :
> : --
> : This message was sent by Atlassian JIRA
> : (v6.3.4#6332)
> :
> : ---------------------------------------------------------------------
> : To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> : For additional commands, e-mail: dev-help@lucene.apache.org
> :
> :
>
> -Hoss
> http://www.lucidworks.com/
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org