Threads in Solr

[Evgeniy Strokin <ev...@yahoo.com>]

Hello,
I'm overwriting getFacetInfo(...) method from standard request handler (BTW: thanks for making a separate method for faceting :-))
What I need is to ran original query several times with filter query which I generate based on result from original query. Bellow is part of my code.
I was thinking may be I could run those queries not one by one but in parallel, in separate threads. But it appears that it takes longer than to run queries one by one.
Do you have any idea why? Do you think the idea to run those queries in separate threads is good in general? Are SolrIndexSearcher and SimpleFacets thread safe?

Thank you
Gene

------------------------------------------------------------------
    protected NamedList getFacetInfo(SolrQueryRequest req,
                                     SolrQueryResponse rsp,
                                     DocSet mainSet) {
        SimpleFacets f = new SimpleFacets(req.getSearcher(),
                mainSet,
                req.getParams());
        NamedList facetInfo = f.getFacetCounts();
////////////////// This is custom code for multi facets
.................. 
........ Truncated 
..................
                    for (int i = 0; i < shortFld.size(); i++) {
                        SolrQueryParser qp = new SolrQueryParser(s.getSchema(), null);
                        Query q = qp.parse(shortFldName + ":" + shortFld.getName(i));
                        filters.add(q);
                        DocListAndSet matrixRes = s.getDocListAndSet(query, filters, null, 0, 0, flags);
                        NamedList matr = new SimpleFacets(req.getSearcher(),
                                matrixRes.docSet,
                                req.getParams()).getFacetCounts();
                        facetFields.add(shortFld.getName(i), matr.get("facet_fields"));
                        
                        filters.remove(q);
                    }
.................. 
........ Truncated 
..................
        return facetInfo;
    }

Re: Threads in Solr

[Chris Hostetter <ho...@fucit.org>]

: I was thinking may be I could run those queries not one by one but in 
: parallel, in separate threads. But it appears that it takes longer than 
: to run queries one by one.

: Do you have any idea why? Do you think the idea to run those queries in 
: separate threads is good in general? Are SolrIndexSearcher and 
: SimpleFacets thread safe?

SolrIndexSearcher is threadsafe ... SimpleFacets should be thread safe, 
but i won't swear to it off the top of my head.  without seeing exactly 
how you setup your threads, it's hard to guess ... in general multiple 
threads are only useful if you are io bound, or have hardware that can 
take advantage of parallelization (ie: multiple cores).

but it's also possible that things take just as long because all of your 
threads wind up computing the same DocSets at the same time -- or block on 
generating the same FieldCache arrays at the same time.



-Hoss