[jira] [Updated] (LUCENE-6294) Generalize how IndexSearcher parallelizes collection execution

["Adrien Grand (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/LUCENE-6294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Adrien Grand updated LUCENE-6294:
---------------------------------
    Attachment: LUCENE-6294.patch

Here is a patch that demonstrates the idea. This does not change any API on Collector since not all collectors could work this way, but adds a CollectorManager object which can create collectors and merge them. I cut over top docs collection to this new API and also added IndexSearcher.count to exercise it.

> Generalize how IndexSearcher parallelizes collection execution
> --------------------------------------------------------------
>
>                 Key: LUCENE-6294
>                 URL: https://issues.apache.org/jira/browse/LUCENE-6294
>             Project: Lucene - Core
>          Issue Type: Improvement
>            Reporter: Adrien Grand
>            Assignee: Adrien Grand
>            Priority: Trivial
>         Attachments: LUCENE-6294.patch
>
>
> IndexSearcher takes an ExecutorService that can be used to parallelize collection execution. This is useful if you want to trade throughput for latency.
> However, this executor service will only be used if you search for top docs. In that case, we will create one collector per slide and call TopDocs.merge in the end. If you use search(Query, Collector), the executor service will never be used.
> But there are other collectors that could work the same way as top docs collectors, eg. TotalHitCountCollector. And maybe also some of our users' collectors. So maybe IndexSearcher could expose a generic way to take advantage of the executor service?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org