[jira] [Commented] (SOLR-2548) Multithreaded faceting

["Hoss Man (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SOLR-2548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13049341#comment-13049341 ] 

Hoss Man commented on SOLR-2548:
--------------------------------

Janne: thanks for the awesome patch!

in general i think this type of functionality is a good idea -- the real question is how it should be configured/controlled.

admins with many CPUs who expect low amounts of concurrent user traffic might be ok with spawning availableProcessors() threads per request, but admins with less CPUs then concurrent requests are going to prefer that individual requests stay single threaded and take a little longer.

The suggestion to use a thread pool based executorservice definitely seems like it makes more sense, then it just becomes a matter of asking the admin to configure a simple number determining the size of the threadpool, we just need to support a few sentinal values: NUM_CPUS, and NONE (always use callers thread).

since we'd want a threadpool that lives longer then a single request, this definitely shouldn't be an option specified via SolrParams (not to mention the risk involved if people don't lock it down with invariants).  That leaves the question of wether this should be an "init" param on the FacetComponent, or something more global.

My first thought was that we should bite the bullet and add a new top level config for a global thread pool executor service that any solr plugin could start using, but after thinking about it some more i think that would not only be premature, but perhaps even a bad idea in general -- even if we assume something like DIH, UIMA, or Highlighting could also take advantage of a shared thread pool owned by solr, if you really care about parallelizing faceting, you probably wouldn't want some other intensive component starving out the thread pool (or vice versa)

> Multithreaded faceting
> ----------------------
>
>                 Key: SOLR-2548
>                 URL: https://issues.apache.org/jira/browse/SOLR-2548
>             Project: Solr
>          Issue Type: Improvement
>          Components: search
>    Affects Versions: 3.1
>            Reporter: Janne Majaranta
>            Priority: Minor
>              Labels: facet
>         Attachments: SOLR-2548.patch, SOLR-2548_for_31x.patch
>
>
> Add multithreading support for faceting.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org