You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by wojtekpia <wo...@hotmail.com> on 2008/12/02 19:10:54 UTC
Re: new faceting algorithm
Is there a configurable way to switch to the previous implementation? I'd
like to see exactly how it affects performance in my case.
Yonik Seeley wrote:
>
> And if you want to verify that the new faceting code has indeed kicked
> in, some statistics are logged, like:
>
> Nov 24, 2008 11:14:32 PM org.apache.solr.request.UnInvertedField uninvert
> INFO: UnInverted multi-valued field features, memSize=14584, time=47,
> phase1=47,
> nTerms=285, bigTerms=99, termInstances=186
>
> -Yonik
>
>
--
View this message in context: http://www.nabble.com/new-faceting-algorithm-tp20674902p20797812.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: new faceting algorithm
Posted by wojtekpia <wo...@hotmail.com>.
It looks like my filterCache was too big. I reduced my filterCache size from
700,000 to 20,000 (without changing the heap size) and all my performance
issues went away. I experimented with various GC settings, but none of them
made a significant difference.
I see a 16% increase in throughput by applying this patch.
Yonik Seeley wrote:
>
> ... This can be a big chunk of memory
> per-request, and is most likely what changed your GC profile (i.e.
> changing the GC settings may help).
>
>
--
View this message in context: http://www.nabble.com/new-faceting-algorithm-tp20674902p20984502.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: new faceting algorithm
Posted by Yonik Seeley <yo...@apache.org>.
On Thu, Dec 4, 2008 at 2:28 PM, wojtekpia <wo...@hotmail.com> wrote:
> I'm seeing some strange behavior with my garbage collector that disappears
> when I turn off this optimization.
I just changed the new faceting code to use a solr cache.
Look for the "fieldValueCache" on the statistics page now.
It just occurred to me that there is a big difference in how memory is
used with facet.method=fc.
Since we traverse documents and count up terms, we need to allocate an
int[nTerms]
to accumulate those counts. This can be a big chunk of memory
per-request, and is most likely what changed your GC profile (i.e.
changing the GC settings may help).
-Yonik
> I'm running load tests on my deployment.
> For the first few minutes, everything is fine (and this patch does make
> things faster - I haven't quantified the improvement yet). After that, the
> garbage collector stops collecting. Specifically, the new generation part of
> the heap is full, but never garbage collected, and the old generation is
> emptied, then never gets anything more. This throttles Solr performance
> (average response times that used to be ~500ms are now ~25s).
>
> I described my deployment scenario in an earlier post:
> http://www.nabble.com/Throughput-Optimization-td20335132.html
>
> Does it sound like the new faceting algorithm could be the culprit?
>
>
> wojtekpia wrote:
>>
>> Definitely, but it'll take me a few days. I'll also report findings on
>> SOLR-465. (I've been on holiday for a few weeks)
>>
>>
>> Noble Paul നോബിള് नोब्ळ् wrote:
>>>
>>> wojtek, you can report back the numbers if possible
>>>
>>> It would be nice to know how the new impl performs in real-world
>>>
>>>
>>>
>>
>>
>
> --
> View this message in context: http://www.nabble.com/new-faceting-algorithm-tp20674902p20840622.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>
>
Re: new faceting algorithm
Posted by Yonik Seeley <yo...@apache.org>.
On Thu, Dec 4, 2008 at 2:57 PM, wojtekpia <wo...@hotmail.com> wrote:
> Yonik Seeley wrote:
>>
>> Are you doing commits at any time?
>> One possibility is the caching mechanism (weak-ref on the
>> IndexReader)... that's going to be changing soon hopefully.
>>
>> -Yonik
>>
>
> No commits during this test. Should I start looking into my heap size
> distribution and garbage collector selection?
Hmmm, OK. The other big difference would then be that retrieving the
top facets requires creating a Lucene TermEnum (not all facet values
are stored in memory). The lucene version in Solr has changed since I
did long running tests... with various Lucene changes to thread-local
caching, etc. I'll try and reproduce. Or maybe this is somehow a GC
bug just tickled by the current caching mechanism? (weak hash map)
-Yonik
Re: new faceting algorithm
Posted by wojtekpia <wo...@hotmail.com>.
Yonik Seeley wrote:
>
>
> Are you doing commits at any time?
> One possibility is the caching mechanism (weak-ref on the
> IndexReader)... that's going to be changing soon hopefully.
>
> -Yonik
>
No commits during this test. Should I start looking into my heap size
distribution and garbage collector selection?
--
View this message in context: http://www.nabble.com/new-faceting-algorithm-tp20674902p20841219.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: new faceting algorithm
Posted by Yonik Seeley <yo...@apache.org>.
On Thu, Dec 4, 2008 at 2:28 PM, wojtekpia <wo...@hotmail.com> wrote:
> I'm seeing some strange behavior with my garbage collector that disappears
> when I turn off this optimization. I'm running load tests on my deployment.
> For the first few minutes, everything is fine (and this patch does make
> things faster - I haven't quantified the improvement yet). After that, the
> garbage collector stops collecting. Specifically, the new generation part of
> the heap is full, but never garbage collected, and the old generation is
> emptied, then never gets anything more.
Are you doing commits at any time?
One possibility is the caching mechanism (weak-ref on the
IndexReader)... that's going to be changing soon hopefully.
-Yonik
> This throttles Solr performance
> (average response times that used to be ~500ms are now ~25s).
>
> I described my deployment scenario in an earlier post:
> http://www.nabble.com/Throughput-Optimization-td20335132.html
>
> Does it sound like the new faceting algorithm could be the culprit?
>
>
> wojtekpia wrote:
>>
>> Definitely, but it'll take me a few days. I'll also report findings on
>> SOLR-465. (I've been on holiday for a few weeks)
>>
>>
>> Noble Paul നോബിള് नोब्ळ् wrote:
>>>
>>> wojtek, you can report back the numbers if possible
>>>
>>> It would be nice to know how the new impl performs in real-world
>>>
>>>
>>>
>>
>>
>
> --
> View this message in context: http://www.nabble.com/new-faceting-algorithm-tp20674902p20840622.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>
>
Re: new faceting algorithm
Posted by wojtekpia <wo...@hotmail.com>.
I'm seeing some strange behavior with my garbage collector that disappears
when I turn off this optimization. I'm running load tests on my deployment.
For the first few minutes, everything is fine (and this patch does make
things faster - I haven't quantified the improvement yet). After that, the
garbage collector stops collecting. Specifically, the new generation part of
the heap is full, but never garbage collected, and the old generation is
emptied, then never gets anything more. This throttles Solr performance
(average response times that used to be ~500ms are now ~25s).
I described my deployment scenario in an earlier post:
http://www.nabble.com/Throughput-Optimization-td20335132.html
Does it sound like the new faceting algorithm could be the culprit?
wojtekpia wrote:
>
> Definitely, but it'll take me a few days. I'll also report findings on
> SOLR-465. (I've been on holiday for a few weeks)
>
>
> Noble Paul നോബിള് नोब्ळ् wrote:
>>
>> wojtek, you can report back the numbers if possible
>>
>> It would be nice to know how the new impl performs in real-world
>>
>>
>>
>
>
--
View this message in context: http://www.nabble.com/new-faceting-algorithm-tp20674902p20840622.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: new faceting algorithm
Posted by wojtekpia <wo...@hotmail.com>.
Definitely, but it'll take me a few days. I'll also report findings on
SOLR-465. (I've been on holiday for a few weeks)
Noble Paul നോബിള് नोब्ळ् wrote:
>
> wojtek, you can report back the numbers if possible
>
> It would be nice to know how the new impl performs in real-world
>
>
>
--
View this message in context: http://www.nabble.com/new-faceting-algorithm-tp20674902p20798456.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: new faceting algorithm
Posted by Noble Paul നോബിള് नोब्ळ् <no...@gmail.com>.
wojtek, you can report back the numbers if possible
It would be nice to know how the new impl performs in real-world
On Tue, Dec 2, 2008 at 11:45 PM, Yonik Seeley <yo...@apache.org> wrote:
> On Tue, Dec 2, 2008 at 1:10 PM, wojtekpia <wo...@hotmail.com> wrote:
>> Is there a configurable way to switch to the previous implementation? I'd
>> like to see exactly how it affects performance in my case.
>
> Thanks for the reminder, I need to document this in the wiki.
>
> facet.method=enum (enumerate terms and do intersections, the old default)
> facet.method=fc (fieldcache method, the new default)
>
> -Yonik
>
>>
>> Yonik Seeley wrote:
>>>
>>> And if you want to verify that the new faceting code has indeed kicked
>>> in, some statistics are logged, like:
>>>
>>> Nov 24, 2008 11:14:32 PM org.apache.solr.request.UnInvertedField uninvert
>>> INFO: UnInverted multi-valued field features, memSize=14584, time=47,
>>> phase1=47,
>>> nTerms=285, bigTerms=99, termInstances=186
>>>
>>> -Yonik
>>>
>>>
>>
>> --
>> View this message in context: http://www.nabble.com/new-faceting-algorithm-tp20674902p20797812.html
>> Sent from the Solr - User mailing list archive at Nabble.com.
>>
>>
>
--
--Noble Paul
Re: new faceting algorithm
Posted by Yonik Seeley <yo...@apache.org>.
On Tue, Dec 2, 2008 at 1:10 PM, wojtekpia <wo...@hotmail.com> wrote:
> Is there a configurable way to switch to the previous implementation? I'd
> like to see exactly how it affects performance in my case.
Thanks for the reminder, I need to document this in the wiki.
facet.method=enum (enumerate terms and do intersections, the old default)
facet.method=fc (fieldcache method, the new default)
-Yonik
>
> Yonik Seeley wrote:
>>
>> And if you want to verify that the new faceting code has indeed kicked
>> in, some statistics are logged, like:
>>
>> Nov 24, 2008 11:14:32 PM org.apache.solr.request.UnInvertedField uninvert
>> INFO: UnInverted multi-valued field features, memSize=14584, time=47,
>> phase1=47,
>> nTerms=285, bigTerms=99, termInstances=186
>>
>> -Yonik
>>
>>
>
> --
> View this message in context: http://www.nabble.com/new-faceting-algorithm-tp20674902p20797812.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>
>