You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Rafał Kuć <r....@solr.pl> on 2011/11/11 22:34:13 UTC
Group.ngroup parameter memory consumption
Hello!
I was wondering if there is a way for calculating the memory
consumption of group.ngroups parameter. I know the the answer can be
'that depends', but what I'm actually wondering about is what the
memory consumption depends on - number of documents returned by a
query or number of groups ? Or maybe by it depends on another factor ?
--
Regards,
Rafał Kuć
Re: Group.ngroup parameter memory consumption
Posted by Martijn v Groningen <ma...@gmail.com>.
BTW this applies for 4.0-dev. In 3x the String instance from a
StringIndex is directly used, this is then put into a list. So there
is no extra object instance created per group matching the query.
Martijn
On 12 November 2011 08:49, Rafał Kuć <r....@solr.pl> wrote:
> Hello!
>
> Thanks, that's what I was looking for :)
>
> --
> Regards,
> Rafał Kuć
> http://solr.pl
>
>> The ngroup option collects per search the number of unique groups
>> matching the query. Based on the collected groups it returns the
>> count.
>> So it depends of the number of groups matching the query.
>> To get more in detail: per unique group a ByteRef instance is created
>> to represent a group and this put into a ArrayList. I think you can
>> use this to roughly calculate the memory used per request.
>
>> Besides this the ngroup relies in most cases on the FieldCache which
>> is also takes a decent amount of memory. But just like sorting this is
>> cache is
>> shared between requests.
>
>> Martijn
>
>> On 11 November 2011 22:34, Rafał Kuć <r....@solr.pl> wrote:
>>> Hello!
>>>
>>> I was wondering if there is a way for calculating the memory
>>> consumption of group.ngroups parameter. I know the the answer can be
>>> 'that depends', but what I'm actually wondering about is what the
>>> memory consumption depends on - number of documents returned by a
>>> query or number of groups ? Or maybe by it depends on another factor ?
>>>
>>>
>>>
>>> --
>>> Regards,
>>> Rafał Kuć
>>>
>>>
>
>
>
>
>
>
>
>
--
Met vriendelijke groet,
Martijn van Groningen
Re: Group.ngroup parameter memory consumption
Posted by Rafał Kuć <r....@solr.pl>.
Hello!
Thanks, that's what I was looking for :)
--
Regards,
Rafał Kuć
http://solr.pl
> The ngroup option collects per search the number of unique groups
> matching the query. Based on the collected groups it returns the
> count.
> So it depends of the number of groups matching the query.
> To get more in detail: per unique group a ByteRef instance is created
> to represent a group and this put into a ArrayList. I think you can
> use this to roughly calculate the memory used per request.
> Besides this the ngroup relies in most cases on the FieldCache which
> is also takes a decent amount of memory. But just like sorting this is
> cache is
> shared between requests.
> Martijn
> On 11 November 2011 22:34, Rafał Kuć <r....@solr.pl> wrote:
>> Hello!
>>
>> I was wondering if there is a way for calculating the memory
>> consumption of group.ngroups parameter. I know the the answer can be
>> 'that depends', but what I'm actually wondering about is what the
>> memory consumption depends on - number of documents returned by a
>> query or number of groups ? Or maybe by it depends on another factor ?
>>
>>
>>
>> --
>> Regards,
>> Rafał Kuć
>>
>>
Re: Group.ngroup parameter memory consumption
Posted by Martijn v Groningen <ma...@gmail.com>.
The ngroup option collects per search the number of unique groups
matching the query. Based on the collected groups it returns the
count.
So it depends of the number of groups matching the query.
To get more in detail: per unique group a ByteRef instance is created
to represent a group and this put into a ArrayList. I think you can
use this to roughly calculate the memory used per request.
Besides this the ngroup relies in most cases on the FieldCache which
is also takes a decent amount of memory. But just like sorting this is
cache is
shared between requests.
Martijn
On 11 November 2011 22:34, Rafał Kuć <r....@solr.pl> wrote:
> Hello!
>
> I was wondering if there is a way for calculating the memory
> consumption of group.ngroups parameter. I know the the answer can be
> 'that depends', but what I'm actually wondering about is what the
> memory consumption depends on - number of documents returned by a
> query or number of groups ? Or maybe by it depends on another factor ?
>
>
>
> --
> Regards,
> Rafał Kuć
>
>
--
Met vriendelijke groet,
Martijn van Groningen