why solr search is slower than lucene so much?

[kafka0102 <ka...@163.com>]

HI.
my solr seach has some performance problem recently.
my query is like that: q=xx&fq=fid:1&fq=atm:[int_time1 TO int_time2],
fid's type is : <fieldType name="int" class="solr.TrieIntField" 
precisionStep="0" omitNorms="true" positionIncrementGap="0"/>
atm's type is : <fieldType name="sint" class="solr.TrieIntField" 
precisionStep="8" omitNorms="true" positionIncrementGap="0"/>
my index's size is about 500M and record num is 3984274.
when I use solr's SolrIndexSearcher.search(QueryResult qr, QueryCommand 
cmd), it cost about70ms. When I changed use lucence'API, just like bottom:

       final SolrQueryRequest req = rb.req;
       final SolrIndexSearcher searcher = req.getSearcher();
       final SolrIndexSearcher.QueryCommand cmd = rb.getQueryCommand();
       final ExecuteTimeStatics timeStatics = 
ExecuteTimeStatics.getExecuteTimeStatics();
       final ExecuteTimeUnit staticUnit = 
timeStatics.addExecuteTimeUnit("test2");
       staticUnit.start();
       final List<Query> query = cmd.getFilterList();
       final BooleanQuery booleanFilter = new BooleanQuery();
       for (final Query q : query) {
         booleanFilter.add(new BooleanClause(q,Occur.MUST));
       }
       booleanFilter.add(new BooleanClause(cmd.getQuery(),Occur.MUST));
       logger.info("q:"+query);
       final Sort sort = cmd.getSort();
       final TopFieldDocs docs = 
searcher.search(booleanFilter,null,20,sort);
       final StringBuilder sbBuilder = new StringBuilder();
       for (final ScoreDoc doc :docs.scoreDocs) {
         sbBuilder.append(doc.doc+",");
       }
       logger.info("hits:"+docs.totalHits+",result:"+sbBuilder.toString());
       staticUnit.end();

it cost only about 20ms.
I'm so confused. For solr's config, I closed cache. For test, I first 
called lucene's, and then solr's.
Maybe I should look solr's source more carefully. But now, can anyone 
knows the reason?

Re: why solr search is slower than lucene so much?

[Yonik Seeley <yo...@lucidimagination.com>]

Careful comparing apples to oranges ;-)
For one, your lucene code doesn't retrieve stored fields.
Did you try the solr request more than once (with a different q, but
the same filters?)

Also, by default, Solr independently caches the filters.  This can be
higher up-front cost, but a win when filters are reused.  If you want
something closer to your lucene code, you could add all the filters to
 the main query and not use "fq".

-Yonik
http://www.lucidimagination.com



On Wed, Oct 20, 2010 at 7:07 AM, kafka0102 <ka...@163.com> wrote:
> HI.
> my solr seach has some performance problem recently.
> my query is like that: q=xx&fq=fid:1&fq=atm:[int_time1 TO int_time2],
> fid's type is : <fieldType name="int" class="solr.TrieIntField"
> precisionStep="0" omitNorms="true" positionIncrementGap="0"/>
> atm's type is : <fieldType name="sint" class="solr.TrieIntField"
> precisionStep="8" omitNorms="true" positionIncrementGap="0"/>
> my index's size is about 500M and record num is 3984274.
> when I use solr's SolrIndexSearcher.search(QueryResult qr, QueryCommand
> cmd), it cost about70ms. When I changed use lucence'API, just like bottom:
>
>      final SolrQueryRequest req = rb.req;
>      final SolrIndexSearcher searcher = req.getSearcher();
>      final SolrIndexSearcher.QueryCommand cmd = rb.getQueryCommand();
>      final ExecuteTimeStatics timeStatics =
> ExecuteTimeStatics.getExecuteTimeStatics();
>      final ExecuteTimeUnit staticUnit =
> timeStatics.addExecuteTimeUnit("test2");
>      staticUnit.start();
>      final List<Query> query = cmd.getFilterList();
>      final BooleanQuery booleanFilter = new BooleanQuery();
>      for (final Query q : query) {
>        booleanFilter.add(new BooleanClause(q,Occur.MUST));
>      }
>      booleanFilter.add(new BooleanClause(cmd.getQuery(),Occur.MUST));
>      logger.info("q:"+query);
>      final Sort sort = cmd.getSort();
>      final TopFieldDocs docs = searcher.search(booleanFilter,null,20,sort);
>      final StringBuilder sbBuilder = new StringBuilder();
>      for (final ScoreDoc doc :docs.scoreDocs) {
>        sbBuilder.append(doc.doc+",");
>      }
>      logger.info("hits:"+docs.totalHits+",result:"+sbBuilder.toString());
>      staticUnit.end();
>
> it cost only about 20ms.
> I'm so confused. For solr's config, I closed cache. For test, I first called
> lucene's, and then solr's.
> Maybe I should look solr's source more carefully. But now, can anyone knows
> the reason?
>
>
>