Grouping performance improvement

[soodyogesh <so...@gmail.com>]

Im facing slow performance for query where im grouping on a field while
querying.

Size of index 57 million records, and we would be targeting 100 million + 

Im using grouping to create category based autosuggest.

so when user press "a"

I go and search for "a" and group by field say products. Now i have noticed
performance of query is really get bad with group by clause.

Im at experimental stage so I can change schema or try other alternative.

Please let me know if there are way to cleverly design your schema to
improve performance  or im meeting some option to fine tune.



--
View this message in context: http://lucene.472066.n3.nabble.com/Grouping-performance-improvement-tp4118549.html
Sent from the Solr - User mailing list archive at Nabble.com.

RE: Grouping performance improvement

[soodyogesh <so...@gmail.com>]

Ok so I cannot move forward with this,

If I use format like q=a&fq=category:(value1 value2 value3)

this gives me results with first category.

What i want is top n results per filter category and I dont want to use
group as performance seems to be very bad for groups my observation is group
query doesn't use cache



--
View this message in context: http://lucene.472066.n3.nabble.com/Grouping-performance-improvement-tp4118549p4120093.html
Sent from the Solr - User mailing list archive at Nabble.com.

RE: Grouping performance improvement

[soodyogesh <so...@gmail.com>]

Thanks Alexey for giving some really good points.

Just to make sure I get it right

Are you suggesting

1. do facets on category first lets say I get 10 distinct category
2. do another query where q=search query and fq= facet category values

May be im missing something, however Im not sure how to get factes along
with lets say 5 documents under each facet value.





--
View this message in context: http://lucene.472066.n3.nabble.com/Grouping-performance-improvement-tp4118549p4118844.html
Sent from the Solr - User mailing list archive at Nabble.com.

RE: Grouping performance improvement

[Alexey Kozhemiakin <Al...@epam.com>]

You can think of using facets by category field instead of grouping. It will be faster and categorization can be done against multiple category fields. Try different facet methods.

If you don't need number of documents in each category and number of unique categories  is relatively low, you might be interested in following performance improvement https://issues.apache.org/jira/browse/SOLR-5725 


Alexey

-----Original Message-----
From: soodyogesh [mailto:soodyogesh@gmail.com] 
Sent: Thursday, February 20, 2014 16:20
To: solr-user@lucene.apache.org
Subject: Grouping performance improvement

Im facing slow performance for query where im grouping on a field while querying.

Size of index 57 million records, and we would be targeting 100 million + 

Im using grouping to create category based autosuggest.

so when user press "a"

I go and search for "a" and group by field say products. Now i have noticed performance of query is really get bad with group by clause.

Im at experimental stage so I can change schema or try other alternative.

Please let me know if there are way to cleverly design your schema to improve performance  or im meeting some option to fine tune.



--
View this message in context: http://lucene.472066.n3.nabble.com/Grouping-performance-improvement-tp4118549.html
Sent from the Solr - User mailing list archive at Nabble.com.