You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Kashif Khan <up...@gmail.com> on 2011/12/08 11:02:29 UTC
How to improve facet search?
I have a query and faceting on one field. i want to improve the speed of
getting this facet please advise me the solutions what are all the options
available to get this facet faster.
--
View this message in context: http://lucene.472066.n3.nabble.com/How-to-improve-facet-search-tp3569910p3569910.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: How to improve facet search?
Posted by Erick Erickson <er...@gmail.com>.
These are really horrible numbers:
hitratio : 0.18
inserts : 5488
evictions : 5011
Your cache isn't doing you much good, part of the problem
is you're faceting on so many values and it's chewing through
your cache. If your full data set has significantly more terms,
you're going to have trouble scaling this solution.
Faceting works best on fields with fewer unique values,
so you're pushing it in a direction that it wasn't really designed
for.
What are you trying to accomplish by faceting on so many values?
Perhaps there's another way to accomplish that end, this might
be an "XY" problem.
Best
Erick
On Thu, Dec 8, 2011 at 8:08 AM, Kashif Khan <up...@gmail.com> wrote:
> I have increased the filtercache values as to as <filterCache
> class="solr.FastLRUCache" size="16384" initialSize="4096"
> autowarmCount="4096"/> and <documentCache class="solr.LRUCache"
> size="16384" initialSize="16384"/> and <queryResultCache
> class="solr.LRUCache" size="16384" initialSize="4096"
> autowarmCount="1024"/> but still it performs as in seconds. I have got
> shards along with an average of 6 gb of data is indexed including the
> shards. But in production servers the data including shards has average of
> 80Gb data. it has 4,273 unique terms after doing facet search for a
> particular query. Are you asking unique terms for *:* query or for some
> particular query? 1 gb of ram is allocated for each shard.
> --
> Kashif Khan. B.E.,
> +91 99805 57379
> http://www.kashifkhan.in
>
>
>
> On Thu, Dec 8, 2011 at 4:53 PM, pravesh [via Lucene] <
> ml-node+s472066n3570048h41@n3.nabble.com> wrote:
>
>> How many unique terms do you have in the faceting field????
>> Since there are lot of evictions, consider increasing the size of the
>> filtercache. Try to keep evictions to min.
>>
>> BTW how much is your index size (GB/MB??) How much RAM is allocated?
>>
>> Above All: Have you benchmarked your search? Is searching done in
>> milis/secs/mins?? I am trying to understand if your search could already be
>> performing quite good/OK.
>>
>> Regards
>> Pravesh
>>
>> ------------------------------
>> If you reply to this email, your message will be added to the discussion
>> below:
>>
>> http://lucene.472066.n3.nabble.com/How-to-improve-facet-search-tp3569910p3570048.html
>> To unsubscribe from How to improve facet search?, click here<http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=3569910&code=dXBsaW5rMjAxMEBnbWFpbC5jb218MzU2OTkxMHwtMTgzODU3NDI3OQ==>
>> .
>> NAML<http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.InstantMailNamespace&breadcrumbs=instant+emails%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>>
>
>
> --
> View this message in context: http://lucene.472066.n3.nabble.com/How-to-improve-facet-search-tp3569910p3570246.html
> Sent from the Solr - User mailing list archive at Nabble.com.
Re: How to improve facet search?
Posted by Kashif Khan <up...@gmail.com>.
I have increased the filtercache values as to as <filterCache
class="solr.FastLRUCache" size="16384" initialSize="4096"
autowarmCount="4096"/> and <documentCache class="solr.LRUCache"
size="16384" initialSize="16384"/> and <queryResultCache
class="solr.LRUCache" size="16384" initialSize="4096"
autowarmCount="1024"/> but still it performs as in seconds. I have got
shards along with an average of 6 gb of data is indexed including the
shards. But in production servers the data including shards has average of
80Gb data. it has 4,273 unique terms after doing facet search for a
particular query. Are you asking unique terms for *:* query or for some
particular query? 1 gb of ram is allocated for each shard.
--
Kashif Khan. B.E.,
+91 99805 57379
http://www.kashifkhan.in
On Thu, Dec 8, 2011 at 4:53 PM, pravesh [via Lucene] <
ml-node+s472066n3570048h41@n3.nabble.com> wrote:
> How many unique terms do you have in the faceting field????
> Since there are lot of evictions, consider increasing the size of the
> filtercache. Try to keep evictions to min.
>
> BTW how much is your index size (GB/MB??) How much RAM is allocated?
>
> Above All: Have you benchmarked your search? Is searching done in
> milis/secs/mins?? I am trying to understand if your search could already be
> performing quite good/OK.
>
> Regards
> Pravesh
>
> ------------------------------
> If you reply to this email, your message will be added to the discussion
> below:
>
> http://lucene.472066.n3.nabble.com/How-to-improve-facet-search-tp3569910p3570048.html
> To unsubscribe from How to improve facet search?, click here<http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=3569910&code=dXBsaW5rMjAxMEBnbWFpbC5jb218MzU2OTkxMHwtMTgzODU3NDI3OQ==>
> .
> NAML<http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.InstantMailNamespace&breadcrumbs=instant+emails%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>
--
View this message in context: http://lucene.472066.n3.nabble.com/How-to-improve-facet-search-tp3569910p3570246.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: How to improve facet search?
Posted by pravesh <su...@yahoo.com>.
How many unique terms do you have in the faceting field????
Since there are lot of evictions, consider increasing the size of the
filtercache. Try to keep evictions to min.
BTW how much is your index size (GB/MB??) How much RAM is allocated?
Above All: Have you benchmarked your search? Is searching done in
milis/secs/mins?? I am trying to understand if your search could already be
performing quite good/OK.
Regards
Pravesh
--
View this message in context: http://lucene.472066.n3.nabble.com/How-to-improve-facet-search-tp3569910p3570048.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: How to improve facet search?
Posted by Kashif Khan <up...@gmail.com>.
filed type is either long or string.
Solr Statistics are:
*name: * filterCache *class: * org.apache.solr.search.FastLRUCache *
version: * 1.0 *description: * Concurrent LRU Cache(maxSize=512,
initialSize=512, minSize=460, acceptableSize=486, cleanupThread=false) *
stats: * lookups : 6679
hits : 1216
hitratio : 0.18
inserts : 5488
evictions : 5011
size : 477
warmupTime : 0
cumulative_lookups : 6679
cumulative_hits : 1216
cumulative_hitratio : 0.18
cumulative_inserts : 5488
cumulative_evictions : 5011
--
Kashif Khan. B.E.,
+91 99805 57379
http://www.kashifkhan.in
On Thu, Dec 8, 2011 at 3:55 PM, pravesh [via Lucene] <
ml-node+s472066n3569955h58@n3.nabble.com> wrote:
> What is the type of the field on which you are getting facets (string,
> Text, int, date etc.). Is it multivalued or not?
> How many unique values do you have for the field?
>
> What is your filtercache setting in your solrconfig.xml?
>
> Regards
> Pravesh
>
> ------------------------------
> If you reply to this email, your message will be added to the discussion
> below:
>
> http://lucene.472066.n3.nabble.com/How-to-improve-facet-search-tp3569910p3569955.html
> To unsubscribe from How to improve facet search?, click here<http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=3569910&code=dXBsaW5rMjAxMEBnbWFpbC5jb218MzU2OTkxMHwtMTgzODU3NDI3OQ==>
> .
> NAML<http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.InstantMailNamespace&breadcrumbs=instant+emails%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>
--
View this message in context: http://lucene.472066.n3.nabble.com/How-to-improve-facet-search-tp3569910p3569978.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: How to improve facet search?
Posted by pravesh <su...@yahoo.com>.
What is the type of the field on which you are getting facets (string, Text,
int, date etc.). Is it multivalued or not?
How many unique values do you have for the field?
What is your filtercache setting in your solrconfig.xml?
Regards
Pravesh
--
View this message in context: http://lucene.472066.n3.nabble.com/How-to-improve-facet-search-tp3569910p3569955.html
Sent from the Solr - User mailing list archive at Nabble.com.