massive memory consumption of grouping feature

[clawu01 <cl...@bertelsmann.de>]

Hello,

we are upgrading solr from 1.3 to 4.0.
In solr 1.3 we used the SOLR-236 patch to realize grouping/ field
collapsing.
We did not have a memory issue with the field collapsing feature in our 1.3
version.
However, we do now. The query looks something like this:

http://localhost:8983/solr/select?fl=*,score&group.ngroups=true&group.limit=-1&group.field=someGroupingField&group=true&fq=someField:someValue&fq=anotherField:anotherValue&wt=xml&fq=thirdField:[0+TO+1]&rows=30000

as you can see the q parameter is empty, but it does not make a difference
if I query for q=someValue+anotherValue

The result returns:
<int name="matches">3772</int>
<int name="ngroups">2175</int>

We have a memory consumption of about 4G. What causes this massive memory
consumption? How can it be reduced?

Regards,

Claas 



--
View this message in context: http://lucene.472066.n3.nabble.com/massive-memory-consumption-of-grouping-feature-tp4031895.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: massive memory consumption of grouping feature

[Furkan KAMACI <fu...@gmail.com>]

Hi Alok;

Please do not reply an old message at mail list. Users may not see the
question. Instead of that start a new thread and give a link to original
one.


2013/10/3 Alok Bhandari <al...@gmail.com>

> Did find any solution to this. I am also facing the same issue.
>
>
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/massive-memory-consumption-of-grouping-feature-tp4031895p4093297.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>

Re: massive memory consumption of grouping feature

[Alok Bhandari <al...@gmail.com>]

Did find any solution to this. I am also facing the same issue.



--
View this message in context: http://lucene.472066.n3.nabble.com/massive-memory-consumption-of-grouping-feature-tp4031895p4093297.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: massive memory consumption of grouping feature

[clawu01 <cl...@bertelsmann.de>]

Hi Otis,

thanks for your reply.
When I disable grouping, everything is fine!

Also, when I limit the rows to return with the rows= Parameter I don't run
into memory problems.
Since I still get the information about the hits:
<int name="matches">3793</int>
<int name="ngroups">2175</int>

I was wondering if the massive memory consumption arises in rendering the
response?

Regards,

Claas



--
View this message in context: http://lucene.472066.n3.nabble.com/massive-memory-consumption-of-grouping-feature-tp4031895p4032126.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: massive memory consumption of grouping feature

[Otis Gospodnetic <ot...@gmail.com>]

Hello Class,

How is the memory consumption when you do not use grouping *at all*,
meaning you start Solr and run all your queries, but with group=false?
Maybe it's not grouping that's the (main) "problem", but simply a Solr
upgrade in general.

Otis
--
Solr & ElasticSearch Support
http://sematext.com/





On Wed, Jan 9, 2013 at 10:50 AM, clawu01 <cl...@bertelsmann.de>wrote:

> Hello,
>
> we are upgrading solr from 1.3 to 4.0.
> In solr 1.3 we used the SOLR-236 patch to realize grouping/ field
> collapsing.
> We did not have a memory issue with the field collapsing feature in our 1.3
> version.
> However, we do now. The query looks something like this:
>
>
> http://localhost:8983/solr/select?fl=*,score&group.ngroups=true&group.limit=-1&group.field=someGroupingField&group=true&fq=someField:someValue&fq=anotherField:anotherValue&wt=xml&fq=thirdField:[0+TO+1]&rows=30000
>
> as you can see the q parameter is empty, but it does not make a difference
> if I query for q=someValue+anotherValue
>
> The result returns:
> <int name="matches">3772</int>
> <int name="ngroups">2175</int>
>
> We have a memory consumption of about 4G. What causes this massive memory
> consumption? How can it be reduced?
>
> Regards,
>
> Claas
>
>
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/massive-memory-consumption-of-grouping-feature-tp4031895.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>