You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Benson Margulies (Created) (JIRA)" <ji...@apache.org> on 2012/02/20 01:46:34 UTC
[jira] [Created] (LUCENE-3803) Using a searcher with an executor
service does not work from within a Callable called by that same executor
service
Using a searcher with an executor service does not work from within a Callable called by that same executor service
-------------------------------------------------------------------------------------------------------------------
Key: LUCENE-3803
URL: https://issues.apache.org/jira/browse/LUCENE-3803
Project: Lucene - Java
Issue Type: Bug
Components: core/search
Affects Versions: 3.5
Reporter: Benson Margulies
Attachments: lucene-tc.tgz
The test case I'm about to attach creates an executor service uses invokeAll to run things from it. The 'things' (Callable) that it runs in turn run Lucene searchers using a searcher that has the very same executor service. This turns out to hit some sort of deadlock related to the executor service queue.
I won't be terribly surprised if this is really a matter of a fundamental limitation of the executor service, but if nothing else this JIRA might provoke a warning in the javadoc. Or, what do I know, maybe there's a way Lucene could interact with the executor service that gets along with this?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Updated] (LUCENE-3803) Using a searcher with an executor
service does not work from within a Callable called by that same executor
service
Posted by "Benson Margulies (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Benson Margulies updated LUCENE-3803:
-------------------------------------
Attachment: lucene-tc.tgz
> Using a searcher with an executor service does not work from within a Callable called by that same executor service
> -------------------------------------------------------------------------------------------------------------------
>
> Key: LUCENE-3803
> URL: https://issues.apache.org/jira/browse/LUCENE-3803
> Project: Lucene - Java
> Issue Type: Bug
> Components: core/search
> Affects Versions: 3.5
> Reporter: Benson Margulies
> Attachments: lucene-tc.tgz
>
>
> The test case I'm about to attach creates an executor service uses invokeAll to run things from it. The 'things' (Callable) that it runs in turn run Lucene searchers using a searcher that has the very same executor service. This turns out to hit some sort of deadlock related to the executor service queue.
> I won't be terribly surprised if this is really a matter of a fundamental limitation of the executor service, but if nothing else this JIRA might provoke a warning in the javadoc. Or, what do I know, maybe there's a way Lucene could interact with the executor service that gets along with this?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (LUCENE-3803) Using a searcher with an executor
service does not work from within a Callable called by that same executor
service
Posted by "Uwe Schindler (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13211622#comment-13211622 ]
Uwe Schindler commented on LUCENE-3803:
---------------------------------------
The problem is that you are using the same fixed thread pool for everything. If you only have 1 or 2 threads in the pool (as noted in your mail) and one of those threads calls searcher.search() from within its own callable, searcher.search will try to execute the sub-searches in its own callables. If the thread pool does not have enough free threads, it will wait for a thread to get available (because another callable finishes). The callable that called searcher.search will of course not finish as it waits for itsself to finish -> deadlock.
The bad idea with thread pools that leads to your problem is: Don't start Callables from within another Callable in the same thread pool. This almost always leads to deadlocks.
> Using a searcher with an executor service does not work from within a Callable called by that same executor service
> -------------------------------------------------------------------------------------------------------------------
>
> Key: LUCENE-3803
> URL: https://issues.apache.org/jira/browse/LUCENE-3803
> Project: Lucene - Java
> Issue Type: Bug
> Components: core/search
> Affects Versions: 3.5
> Reporter: Benson Margulies
> Attachments: lucene-tc.tgz
>
>
> The test case I'm about to attach creates an executor service uses invokeAll to run things from it. The 'things' (Callable) that it runs in turn run Lucene searchers using a searcher that has the very same executor service. This turns out to hit some sort of deadlock related to the executor service queue.
> I won't be terribly surprised if this is really a matter of a fundamental limitation of the executor service, but if nothing else this JIRA might provoke a warning in the javadoc. Or, what do I know, maybe there's a way Lucene could interact with the executor service that gets along with this?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (LUCENE-3803) Using a searcher with an executor
service does not work from within a Callable called by that same executor
service
Posted by "Robert Muir (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13211624#comment-13211624 ]
Robert Muir commented on LUCENE-3803:
-------------------------------------
This is not really related to your problem (somewhat) but I also looked at the code:
I'm not sure how many queries you are running at once, but for a 'count' operation I'm not sure
I would even pass a executor service... unless maybe your queries are really intensive to compute
if a document is a match or not.
You would have to test: it might be better to just run single-threaded searches for counting in
your case, especially if you have multiple threads querying the index at once anyway.
> Using a searcher with an executor service does not work from within a Callable called by that same executor service
> -------------------------------------------------------------------------------------------------------------------
>
> Key: LUCENE-3803
> URL: https://issues.apache.org/jira/browse/LUCENE-3803
> Project: Lucene - Java
> Issue Type: Bug
> Components: core/search
> Affects Versions: 3.5
> Reporter: Benson Margulies
> Attachments: lucene-tc.tgz
>
>
> The test case I'm about to attach creates an executor service uses invokeAll to run things from it. The 'things' (Callable) that it runs in turn run Lucene searchers using a searcher that has the very same executor service. This turns out to hit some sort of deadlock related to the executor service queue.
> I won't be terribly surprised if this is really a matter of a fundamental limitation of the executor service, but if nothing else this JIRA might provoke a warning in the javadoc. Or, what do I know, maybe there's a way Lucene could interact with the executor service that gets along with this?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Updated] (LUCENE-3803) Using a searcher with an executor
service does not work from within a Callable called by that same executor
service
Posted by "Benson Margulies (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Benson Margulies updated LUCENE-3803:
-------------------------------------
Description:
The test case I'm about to attach creates an executor service uses invokeAll to run things from it. The 'things' (Callable) that it runs in turn run Lucene searchers using a searcher that has the very same executor service. This turns out to hit some sort of deadlock related to the executor service queue.
I won't be terribly surprised if this is really a matter of a fundamental limitation of the executor service, but if nothing else this JIRA might provoke a warning in the javadoc. Or, what do I know, maybe there's a way Lucene could interact with the executor service that gets along with this?
was:
The test case I'm about to attach creates an executor service uses invokeAll to run things from it. The 'things' (Callable) that it runs in turn run Lucene searchers using a searcher that has the very same executor service. This turns out to hit some sort of deadlock related to the executor service queue.
I won't be terribly surprised if this is really a matter of a fundamental limitation of the executor service, but if nothing else this JIRA might provoke a warning in the javadoc. Or, what do I know, maybe there's a way Lucene could interact with the executor service that gets along with this?
> Using a searcher with an executor service does not work from within a Callable called by that same executor service
> -------------------------------------------------------------------------------------------------------------------
>
> Key: LUCENE-3803
> URL: https://issues.apache.org/jira/browse/LUCENE-3803
> Project: Lucene - Java
> Issue Type: Bug
> Components: core/search
> Affects Versions: 3.5
> Reporter: Benson Margulies
> Attachments: lucene-tc.tgz
>
>
> The test case I'm about to attach creates an executor service uses invokeAll to run things from it. The 'things' (Callable) that it runs in turn run Lucene searchers using a searcher that has the very same executor service. This turns out to hit some sort of deadlock related to the executor service queue.
> I won't be terribly surprised if this is really a matter of a fundamental limitation of the executor service, but if nothing else this JIRA might provoke a warning in the javadoc. Or, what do I know, maybe there's a way Lucene could interact with the executor service that gets along with this?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org