You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Shalin Shekhar Mangar (JIRA)" <ji...@apache.org> on 2014/02/18 07:37:20 UTC
[jira] [Commented] (SOLR-1880) Performance: Distributed Search
should skip GET_FIELDS stage if EXECUTE_QUERY stage gets all fields
[ https://issues.apache.org/jira/browse/SOLR-1880?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13903824#comment-13903824 ]
Shalin Shekhar Mangar commented on SOLR-1880:
---------------------------------------------
bq. Also, the "fl" param only gets id, score and sort columns, it would probably be cheaper to fetch the final sort column data in STAGE_EXECUTE_QUERY which has to read the sort column data anyway, and skip STAGE_GET_FIELDS.
Thanks Vitaliy. When fl=id,score,sortField then the STAGE_GET_FIELDS is still executed, right? In other words, the only case which is optimized is when fl=id,score. That alone is also a nice improvement but since the issue description as well as your test has the above comment, I thought I should ask.
> Performance: Distributed Search should skip GET_FIELDS stage if EXECUTE_QUERY stage gets all fields
> ---------------------------------------------------------------------------------------------------
>
> Key: SOLR-1880
> URL: https://issues.apache.org/jira/browse/SOLR-1880
> Project: Solr
> Issue Type: Improvement
> Components: search
> Affects Versions: 1.4
> Reporter: Shawn Smith
> Assignee: Shalin Shekhar Mangar
> Attachments: ASF.LICENSE.NOT.GRANTED--one-pass-query-v1.4.0.patch, ASF.LICENSE.NOT.GRANTED--one-pass-query.patch, SOLR-1880.patch
>
>
> Right now, a typical distributed search using QueryComponent makes two HTTP requests to each shard:
> # STAGE_EXECUTE_QUERY executes one HTTP request to each shard to get top N ids and sort keys, merges the results to produce a final list of document IDs (PURPOSE_GET_TOP_IDS).
> # STAGE_GET_FIELDS executes a second HTTP request to each shard to get the document field values for the final list of document IDs (PURPOSE_GET_FIELDS).
> If the "fl" param is just "id" or just "id,score", all document data to return is already fetched by STAGE_EXECUTE_QUERY. The second STAGE_GET_FIELDS query is completely unnecessary. Eliminating that 2nd HTTP request can make a big difference in overall performance.
> Also, the "fl" param only gets id, score and sort columns, it would probably be cheaper to fetch the final sort column data in STAGE_EXECUTE_QUERY which has to read the sort column data anyway, and skip STAGE_GET_FIELDS.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: [jira] [Commented] (SOLR-1880) Performance: Distributed Search
should skip GET_FIELDS stage if EXECUTE_QUERY stage gets all fields
Posted by mike_大雄 <42...@qq.com>.
haha
JIRA jira@apache.org wrote
> [
> https://issues.apache.org/jira/browse/SOLR-1880?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13903824#comment-13903824
> ]
>
> Shalin Shekhar Mangar commented on SOLR-1880:
> ---------------------------------------------
>
> bq. Also, the "fl" param only gets id, score and sort columns, it would
> probably be cheaper to fetch the final sort column data in
> STAGE_EXECUTE_QUERY which has to read the sort column data anyway, and
> skip STAGE_GET_FIELDS.
>
> Thanks Vitaliy. When fl=id,score,sortField then the STAGE_GET_FIELDS is
> still executed, right? In other words, the only case which is optimized is
> when fl=id,score. That alone is also a nice improvement but since the
> issue description as well as your test has the above comment, I thought I
> should ask.
>
>> Performance: Distributed Search should skip GET_FIELDS stage if
>> EXECUTE_QUERY stage gets all fields
>> ---------------------------------------------------------------------------------------------------
>>
>> Key: SOLR-1880
>> URL: https://issues.apache.org/jira/browse/SOLR-1880
>> Project: Solr
>> Issue Type: Improvement
>> Components: search
>> Affects Versions: 1.4
>> Reporter: Shawn Smith
>> Assignee: Shalin Shekhar Mangar
>> Attachments:
>> ASF.LICENSE.NOT.GRANTED--one-pass-query-v1.4.0.patch,
>> ASF.LICENSE.NOT.GRANTED--one-pass-query.patch, SOLR-1880.patch
>>
>>
>> Right now, a typical distributed search using QueryComponent makes two
>> HTTP requests to each shard:
>> # STAGE_EXECUTE_QUERY executes one HTTP request to each shard to get top
>> N ids and sort keys, merges the results to produce a final list of
>> document IDs (PURPOSE_GET_TOP_IDS).
>> # STAGE_GET_FIELDS executes a second HTTP request to each shard to get
>> the document field values for the final list of document IDs
>> (PURPOSE_GET_FIELDS).
>> If the "fl" param is just "id" or just "id,score", all document data to
>> return is already fetched by STAGE_EXECUTE_QUERY. The second
>> STAGE_GET_FIELDS query is completely unnecessary. Eliminating that 2nd
>> HTTP request can make a big difference in overall performance.
>> Also, the "fl" param only gets id, score and sort columns, it would
>> probably be cheaper to fetch the final sort column data in
>> STAGE_EXECUTE_QUERY which has to read the sort column data anyway, and
>> skip STAGE_GET_FIELDS.
>
>
>
> --
> This message was sent by Atlassian JIRA
> (v6.1.5#6160)
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail:
> dev-unsubscribe@.apache
> For additional commands, e-mail:
> dev-help@.apache
--
View this message in context: http://lucene.472066.n3.nabble.com/jira-Commented-SOLR-1880-Performance-Distributed-Search-should-skip-GET-FIELDS-stage-if-EXECUTE-QUERs-tp4117950p4117951.html
Sent from the Lucene - Java Developer mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org