You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Benson Margulies <bi...@gmail.com> on 2012/04/12 02:15:23 UTC
http://wiki.apache.org/solr/WritingDistributedSearchComponents
I'd like to add some more information to this; to that end, I'd
appreciate a correctness review of the following:
* QueryComponent distribution algorithm
The query component distributes queries in two passes. First, it sends
the query out to the shards with the field list trimmed to be just the
score and the doc ID. Upon receipt of these results, it merges the
lists, and then issues new queries out to the shards with
ids=id1,id2,... to retrieve the remaining fields. The results are
packaged up and flow out through the result writer.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: http://wiki.apache.org/solr/WritingDistributedSearchComponents
Posted by Li Li <fa...@gmail.com>.
me too. maybe it should provide a one-stage component.
On Fri, Apr 13, 2012 at 1:41 AM, Mikhail Khludnev <
mkhludnev@griddynamics.com> wrote:
> Btw, I always wanted to ask why it's always done in two stages. It seems
> to me that it's purposed for the specific usecase. But shouldn't we have an
> option to do it in the single stage? eg. if I have a few fields to retrieve
> and not really deep paging, I'd like to retrieve whole fl-s and merge them
> by PKs.
>
> Benson,
> Thanks for contributing the doc for community. I spend a lot of time to
> get into that machinery on my own.
>
> Regards
>
>
> On Thu, Apr 12, 2012 at 4:17 AM, Yonik Seeley <yo...@lucidimagination.com>wrote:
>
>> On Wed, Apr 11, 2012 at 8:15 PM, Benson Margulies <bi...@gmail.com>
>> wrote:
>> > I'd like to add some more information to this; to that end, I'd
>> > appreciate a correctness review of the following:
>> >
>> > * QueryComponent distribution algorithm
>> >
>> > The query component distributes queries in two passes. First, it sends
>> > the query out to the shards with the field list trimmed to be just the
>> > score and the doc ID. Upon receipt of these results, it merges the
>> > lists, and then issues new queries out to the shards with
>> > ids=id1,id2,... to retrieve the remaining fields. The results are
>> > packaged up and flow out through the result writer.
>>
>> Yep.
>>
>> -Yonik
>> lucenerevolution.com - Lucene/Solr Open Source Search Conference.
>> Boston May 7-10
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: dev-help@lucene.apache.org
>>
>>
>
>
> --
> Sincerely yours
> Mikhail Khludnev
> gedel@yandex.ru
>
> <http://www.griddynamics.com>
> <mk...@griddynamics.com>
>
>
Re: http://wiki.apache.org/solr/WritingDistributedSearchComponents
Posted by Mikhail Khludnev <mk...@griddynamics.com>.
Btw, I always wanted to ask why it's always done in two stages. It seems to
me that it's purposed for the specific usecase. But shouldn't we have an
option to do it in the single stage? eg. if I have a few fields to retrieve
and not really deep paging, I'd like to retrieve whole fl-s and merge them
by PKs.
Benson,
Thanks for contributing the doc for community. I spend a lot of time to get
into that machinery on my own.
Regards
On Thu, Apr 12, 2012 at 4:17 AM, Yonik Seeley <yo...@lucidimagination.com>wrote:
> On Wed, Apr 11, 2012 at 8:15 PM, Benson Margulies <bi...@gmail.com>
> wrote:
> > I'd like to add some more information to this; to that end, I'd
> > appreciate a correctness review of the following:
> >
> > * QueryComponent distribution algorithm
> >
> > The query component distributes queries in two passes. First, it sends
> > the query out to the shards with the field list trimmed to be just the
> > score and the doc ID. Upon receipt of these results, it merges the
> > lists, and then issues new queries out to the shards with
> > ids=id1,id2,... to retrieve the remaining fields. The results are
> > packaged up and flow out through the result writer.
>
> Yep.
>
> -Yonik
> lucenerevolution.com - Lucene/Solr Open Source Search Conference.
> Boston May 7-10
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
>
--
Sincerely yours
Mikhail Khludnev
gedel@yandex.ru
<http://www.griddynamics.com>
<mk...@griddynamics.com>
Re: http://wiki.apache.org/solr/WritingDistributedSearchComponents
Posted by Yonik Seeley <yo...@lucidimagination.com>.
On Wed, Apr 11, 2012 at 8:15 PM, Benson Margulies <bi...@gmail.com> wrote:
> I'd like to add some more information to this; to that end, I'd
> appreciate a correctness review of the following:
>
> * QueryComponent distribution algorithm
>
> The query component distributes queries in two passes. First, it sends
> the query out to the shards with the field list trimmed to be just the
> score and the doc ID. Upon receipt of these results, it merges the
> lists, and then issues new queries out to the shards with
> ids=id1,id2,... to retrieve the remaining fields. The results are
> packaged up and flow out through the result writer.
Yep.
-Yonik
lucenerevolution.com - Lucene/Solr Open Source Search Conference.
Boston May 7-10
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org