You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Gregg Donovan <gr...@gmail.com> on 2014/04/14 23:17:01 UTC
Estimating RAM usage of SolrCache instances?
We'd like to graph the approximate RAM size of our SolrCache instances. Our
first attempt at doing this was to use the Lucene RamUsageEstimator [1].
Unfortunately, this appears to give a bogus result. Every instance of
FastLRUCache was judged to have the same exact size, down to the byte. I
assume this is due to an issue with how the variably-sized backing maps
were calculated, but I'm not sure.
Any ideas for how to get an accurate RAM estimation for SolrCache objects?
--Gregg
[1] https://gist.github.com/greggdonovan/10682810
Re: Estimating RAM usage of SolrCache instances?
Posted by Gregg Donovan <gr...@gmail.com>.
Ideally we could get good approximates for all of them, including any of
our custom caches (of which we have about five). The RAM size estimator
spreadsheet [1] is helpful but we'd love to get accurate live size metrics.
[1]
https://github.com/apache/lucene-solr/blob/trunk/dev-tools/size-estimator-lucene-solr.xls
On Mon, Apr 14, 2014 at 10:56 PM, Erick Erickson <er...@gmail.com>wrote:
> _which_ solrCache objects? filterCache? result cache? documentcache?
>
> result cache is about "average size of a query" + "window size *
> sizeof int) for each entry.
> filter cache is about "average size of a filter query" + maxdoc/8
> document cacha is about "average size of the stored fields in bytes" *
> size.
>
> HTH,
> Erick
>
> On Mon, Apr 14, 2014 at 5:17 PM, Gregg Donovan <gr...@gmail.com> wrote:
> > We'd like to graph the approximate RAM size of our SolrCache instances.
> Our
> > first attempt at doing this was to use the Lucene RamUsageEstimator [1].
> >
> > Unfortunately, this appears to give a bogus result. Every instance of
> > FastLRUCache was judged to have the same exact size, down to the byte. I
> > assume this is due to an issue with how the variably-sized backing maps
> > were calculated, but I'm not sure.
> >
> > Any ideas for how to get an accurate RAM estimation for SolrCache
> objects?
> >
> > --Gregg
> >
> > [1] https://gist.github.com/greggdonovan/10682810
>
Re: Estimating RAM usage of SolrCache instances?
Posted by Erick Erickson <er...@gmail.com>.
_which_ solrCache objects? filterCache? result cache? documentcache?
result cache is about "average size of a query" + "window size *
sizeof int) for each entry.
filter cache is about "average size of a filter query" + maxdoc/8
document cacha is about "average size of the stored fields in bytes" * size.
HTH,
Erick
On Mon, Apr 14, 2014 at 5:17 PM, Gregg Donovan <gr...@gmail.com> wrote:
> We'd like to graph the approximate RAM size of our SolrCache instances. Our
> first attempt at doing this was to use the Lucene RamUsageEstimator [1].
>
> Unfortunately, this appears to give a bogus result. Every instance of
> FastLRUCache was judged to have the same exact size, down to the byte. I
> assume this is due to an issue with how the variably-sized backing maps
> were calculated, but I'm not sure.
>
> Any ideas for how to get an accurate RAM estimation for SolrCache objects?
>
> --Gregg
>
> [1] https://gist.github.com/greggdonovan/10682810