You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by ku3ia <de...@gmail.com> on 2014/01/08 19:24:24 UTC
Solr 4.6.0: DocValues (distributed search)
Hi!
https://issues.apache.org/jira/browse/SOLR-3855
Description
It would be nice if Solr supported DocValues:
for ID fields (fewer disk seeks when running distributed search),
Does docValues completely done for distributed search? for ID fields?
P.S. I'm using Solr 4.6.0 not in Cloud.
--
View this message in context: http://lucene.472066.n3.nabble.com/Solr-4-6-0-DocValues-distributed-search-tp4110289.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr 4.6.0: DocValues (distributed search)
Posted by Shawn Heisey <so...@elyograg.org>.
On 1/8/2014 11:24 AM, ku3ia wrote:
> Hi!
> https://issues.apache.org/jira/browse/SOLR-3855
> Description
> It would be nice if Solr supported DocValues:
> for ID fields (fewer disk seeks when running distributed search),
>
> Does docValues completely done for distributed search? for ID fields?
> P.S. I'm using Solr 4.6.0 not in Cloud.
I would like to know the answer to this question as well. I went ahead
and added docValues to my uniqueKey field just in case. If it's not
offering a performance increase now, it likely will in the future.
Thanks,
Shawn
Re: Solr 4.6.0: DocValues (distributed search)
Posted by ku3ia <de...@gmail.com>.
Manuel Le Normand wrote
> In short, when running a distributed search every shard runs the query
> separately. Each shard's collector returns the topN (rows param) internal
> docId's of the matching documents.
>
> These topN docId's are converted to their uniqueKey in the
> BinaryResponseWriter and sent to the frontend core (the one the received
> the query). This conversion is implemented by a StoredFieldVisitor,
> meaning
> the uniqueKeys are read from their stored field and not from their
> docValues.
>
> As in our use-case we have a high row param, these conversions became a
> performance bottleneck. We implemented a user-cache that stores the
> shard's
> uniqueKey docValues, which is a [docId, uniqueKey] mapping. This
> eliminates
> the need of accessing the stored field for these frequent conversions.
>
> You can have a look at the patch. Feel free commenting
> https://issues.apache.org/jira/browse/SOLR-5478
>
> Best,
> Manuel
>
>
> On Thu, Jan 9, 2014 at 7:33 PM, ku3ia <
> demesg@
> > wrote:
>
>> Today I setup a simple SolrCloud with tow shards. Seems the same. When
>> I'm
>> debugging a distributed search I can't catch a break-point at lucene
>> codec
>> file, but when I'm using faceted search everything looks fine - debugger
>> stops.
>>
>> Can anyone help me with my question? Thanks.
>>
>>
>>
>> --
>> View this message in context:
>> http://lucene.472066.n3.nabble.com/Solr-4-6-0-DocValues-distributed-search-tp4110289p4110511.html
>> Sent from the Solr - User mailing list archive at Nabble.com.
>>
Hi, Manuel! Many thanks for your post! I'll try yours patch.
--
View this message in context: http://lucene.472066.n3.nabble.com/Solr-4-6-0-DocValues-distributed-search-tp4110289p4110698.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr 4.6.0: DocValues (distributed search)
Posted by Manuel Le Normand <ma...@gmail.com>.
In short, when running a distributed search every shard runs the query
separately. Each shard's collector returns the topN (rows param) internal
docId's of the matching documents.
These topN docId's are converted to their uniqueKey in the
BinaryResponseWriter and sent to the frontend core (the one the received
the query). This conversion is implemented by a StoredFieldVisitor, meaning
the uniqueKeys are read from their stored field and not from their
docValues.
As in our use-case we have a high row param, these conversions became a
performance bottleneck. We implemented a user-cache that stores the shard's
uniqueKey docValues, which is a [docId, uniqueKey] mapping. This eliminates
the need of accessing the stored field for these frequent conversions.
You can have a look at the patch. Feel free commenting
https://issues.apache.org/jira/browse/SOLR-5478
Best,
Manuel
On Thu, Jan 9, 2014 at 7:33 PM, ku3ia <de...@gmail.com> wrote:
> Today I setup a simple SolrCloud with tow shards. Seems the same. When I'm
> debugging a distributed search I can't catch a break-point at lucene codec
> file, but when I'm using faceted search everything looks fine - debugger
> stops.
>
> Can anyone help me with my question? Thanks.
>
>
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/Solr-4-6-0-DocValues-distributed-search-tp4110289p4110511.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>
Re: Solr 4.6.0: DocValues (distributed search)
Posted by ku3ia <de...@gmail.com>.
Today I setup a simple SolrCloud with tow shards. Seems the same. When I'm
debugging a distributed search I can't catch a break-point at lucene codec
file, but when I'm using faceted search everything looks fine - debugger
stops.
Can anyone help me with my question? Thanks.
--
View this message in context: http://lucene.472066.n3.nabble.com/Solr-4-6-0-DocValues-distributed-search-tp4110289p4110511.html
Sent from the Solr - User mailing list archive at Nabble.com.